ThePlantGame: Actively Training Human Annotators for Domain-specific Crowdsourcing

Maximilien Servajean 1 Alexis Joly 1 Dennis Shasha 2 Julien Champ 1 Esther Pacitti 1
1 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : In a typical citizen science/crowdsourcing environment, the contributors label items. When there are few labels, it is straightforward to train contributors and judge the quality of their labels by giving a few examples with known answers. Neither is true when there are thousands of domain-specic labels and annotators with heterogeneous skills. This demo paper presents an Active User Training framework implemented as a serious game called The- PlantGame. It is based on a set of data-driven algorithms allowing to (i) actively train annotators, and (ii) evaluate the quality of contributors’ answers on new test items to optimize predictions.
Complete list of metadatas

Cited literature [6 references]  Display  Hide  Download

https://hal.inria.fr/hal-01373769
Contributor : Alexis Joly <>
Submitted on : Monday, October 3, 2016 - 12:06:26 PM
Last modification on : Tuesday, September 17, 2019 - 10:38:28 AM
Long-term archiving on: Friday, February 3, 2017 - 1:51:49 PM

File

TPG-demo-ACMM.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Maximilien Servajean, Alexis Joly, Dennis Shasha, Julien Champ, Esther Pacitti. ThePlantGame: Actively Training Human Annotators for Domain-specific Crowdsourcing. MM: Conference on Multimedia, Oct 2016, Amsterdam, Netherlands. pp.720-721, ⟨10.1145/2964284.2973820⟩. ⟨hal-01373769⟩

Share

Metrics

Record views

1125

Files downloads

168