Cost and Quality in Crowdsourcing Workflows - Archive ouverte HAL Access content directly
Conference Papers Year : 2021

Cost and Quality in Crowdsourcing Workflows

(1) , (2) , (2)


Crowdsourcing platforms provide tools to replicate and distribute micro tasks (simple, independent work units) to crowds and assemble results. However, real-life problems are often complex: they require to collect, organize or transform data, with quality and costs constraints. This work considers dynamic realization policies for complex crowdsourcing tasks. Workflows provide ways to organize a complex task in phases and guide its realization. The challenge is then to deploy a workflow on a crowd, i.e., allocate workers to phases so that the overall workflow terminates, with good accuracy of results and at a reasonable cost. Standard "static" allocation of work in crowdsourcing affects a fixed number of workers per micro-task to realize and aggregates the results. We define new dynamic worker allocation techniques that consider progress in a workflow, quality of synthesized data, and remaining budget. Evaluation on a benchmark shows that dynamic approaches outperform static ones in terms of cost and accuracy.
Fichier principal
Vignette du fichier
PN-workflows.pdf (890.92 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03482424 , version 1 (15-12-2021)



Loïc Hélouët, Zoltan Miklos, Rituraj Singh. Cost and Quality in Crowdsourcing Workflows. PETRI NETS 2021 - 42nd International Conference on Applications and Theory of Petri Nets and Concurrency, Jun 2021, Paris, France. pp.33-54, ⟨10.1007/978-3-030-76983-3_3⟩. ⟨hal-03482424⟩
12 View
49 Download



Gmail Facebook Twitter LinkedIn More