Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Reducing the Cost of Aggregation in Crowdsourcing

Rituraj Singh 1 Loïc Hélouët 2 Zoltán Miklós 1
2 SUMO - SUpervision of large MOdular and distributed systems
Inria Rennes – Bretagne Atlantique , IRISA-D4 - LANGAGE ET GÉNIE LOGICIEL
Abstract : Crowdsourcing is a way to solve problems that need human contribution. Crowdsourcing platforms distribute replicated tasks to workers, pay them for their contribution, and aggregate answers to produce a reliable conclusion. A fundamental problem is to infer a correct answer from the set of returned results. Another challenge is to obtain a reliable answer at a reasonable cost: unlimited budget allows hiring experts or large pools of workers for each task but a limited budget forces to use resources at best. This paper considers crowdsourcing of simple boolean tasks. We first define a probabilistic inference technique, that considers difficulty of tasks and expertise of workers when aggregating answers. We then propose CrowdInc, a greedy algorithm that reduce the cost needed to reach a consensual answer. CrowdInc distributes resources dynamically to tasks according to their difficulty. We show on several benchmarks that CrowdInc achieves good accuracy, reduces costs, and we compare its performance to existing solutions.
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.inria.fr/hal-02397971
Contributor : Loic Helouet <>
Submitted on : Friday, December 6, 2019 - 6:02:49 PM
Last modification on : Wednesday, October 14, 2020 - 3:53:23 AM
Long-term archiving on: : Saturday, March 7, 2020 - 4:55:40 PM

File

Main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02397971, version 1

Citation

Rituraj Singh, Loïc Hélouët, Zoltán Miklós. Reducing the Cost of Aggregation in Crowdsourcing. 2019. ⟨hal-02397971⟩

Share

Metrics

Record views

98

Files downloads

321