Skip to Main content Skip to Navigation

Using Hierarchical Skills for Optimized Task Selection in Crowdsourcing

Abstract : A large number of participative applications rely on a crowd to acquire and process data. These participative applications are widely known as crowdsourcing platforms, where amateur enthusiasts are involved in real scientific or commercial projects that requesters have posted online. Most well-known commercial crowdsourcing platforms are Amazon MTurk and Crowdflower. Participants there, select and perform tasks, called microtasks and accept a micropayment in return. Common challenges for such platforms are related to the quality of the required answers, the expertise of the involved crowd, the ease of finding tasks and the respect of tasks’ deadlines. Related work focuses on modeling skills as keywords to improve quality while in this work we formalize skills using a hierarchical structure, that can help substituting tasks with similar skills and take advantage of the whole workforce. With extensive synthetic and real datasets, we show a significant improvement in quality when using a hierarchical structure of skills instead of pure keywords. We also extend our work to study the impact of a participant’s choice given a list of tasks. While our previous solution focused on improving an overall one-to-one matching for tasks and participants, we also examine how participants can choose from a ranked list of tasks. Selecting from an enormous list of tasks can be challenging, time-consuming and affects the quality of answers to crowdsourcing platforms. Existing related work concerning crowdsourcing uses neither a taxonomy nor ranking methods to assist participants. We propose a new model that provides the participant with a short list of tasks. This short list takes into account the diversity of the participant’s skills and the task deadlines as well. Our extensive synthetic and real experiments show that we can meet deadlines, get high-quality answers and keep the interest of participants high while giving them a choice of well-selected tasks.
Keywords : crowdsourcing
Document type :
Complete list of metadata

Cited literature [140 references]  Display  Hide  Download
Contributor : Zoltan Miklos Connect in order to contact the contributor
Submitted on : Friday, March 6, 2020 - 5:10:06 PM
Last modification on : Wednesday, November 3, 2021 - 8:14:23 AM
Long-term archiving on: : Sunday, June 7, 2020 - 4:01:34 PM


Files produced by the author(s)


  • HAL Id : tel-02501323, version 1


Panagiotis Mavridis. Using Hierarchical Skills for Optimized Task Selection in Crowdsourcing. Web. Université de Rennes 1 [UR1], 2017. English. ⟨tel-02501323⟩



Record views


Files downloads