Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis

Abstract : We present a system that lets analysts use paid crowd workers to explore data sets and helps analysts interactively examine and build upon workers' insights. We take advantage of the fact that, for many types of data, independent crowd workers can readily perform basic analysis tasks like examining views and generating explanations for trends and patterns. However, workers operating in parallel can often generate redundant explanations. Moreover, because workers have different competencies and domain knowledge, some responses are likely to be more plausible than others. To efficiently utilize the crowd's work, analysts must be able to quickly identify and consolidate redundant responses and determine which explanations are the most plausible. In this paper, we demonstrate several crowd-assisted techniques to help analysts make better use of crowdsourced explanations: (1) We explore crowd-assisted strategies that utilize multiple workers to detect redundant explanations. We introduce color clustering with representative selection--a strategy in which multiple workers cluster explanations and we automatically select the most-representative result--and show that it generates clusterings that are as good as those produced by experts. (2) We capture explanation provenance by introducing highlighting tasks and capturing workers' browsing behavior via an embedded web browser, and refine that provenance information via source-review tasks. We expose this information in an explanation-management interface that allows analysts to interactively filter and sort responses, select the most plausible explanations, and decide which to explore further.
Type de document :
Article dans une revue
IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2013, 19 (12), pp.2198 - 2206. <10.1109/TVCG.2013.164>
Liste complète des métadonnées


https://hal.inria.fr/hal-00850827
Contributeur : Wesley Willett <>
Soumis le : vendredi 11 octobre 2013 - 07:00:17
Dernière modification le : jeudi 9 février 2017 - 15:50:06
Document(s) archivé(s) le : mercredi 5 avril 2017 - 20:26:16

Fichiers

CrowdAnalytics-VAST2013-FinalC...
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Wesley Willett, Shiry Ginosar, Avital Steinitz, Björn Hartmann, Maneesh Agrawala. Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis. IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2013, 19 (12), pp.2198 - 2206. <10.1109/TVCG.2013.164>. <hal-00850827>

Partager

Métriques

Consultations de
la notice

663

Téléchargements du document

258