Skip to Main content Skip to Navigation
Conference papers

Evidence Humans Provide When Explaining Data-Labeling Decisions

Abstract : Because machine learning would benefit from reduced data requirements, some prior work has proposed using humans not just to label data, but also to explain those labels. To characterize the evidence humans might want to provide, we conducted a user study and a data experiment. In the user study, 75 participants provided classification labels for 20 photos, justifying those labels with free-text explanations. Explanations frequently referenced concepts (objects and attributes) in the image, yet 26% of explanations invoked concepts not in the image. Boolean logic was common in implicit form, but was rarely explicit. In a follow-up experiment on the Visual Genome dataset, we found that some concepts could be partially defined through their relationship to frequently co-occurring concepts, rather than only through labeling.
Document type :
Conference papers
Complete list of metadata

Cited literature [21 references]  Display  Hide  Download
Contributor : Hal Ifip <>
Submitted on : Friday, April 24, 2020 - 6:08:41 PM
Last modification on : Friday, April 24, 2020 - 6:39:47 PM


 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2022-01-01

Please log in to resquest access to the document


Distributed under a Creative Commons Attribution 4.0 International License



Judah Newman, Bowen Wang, Valerie Zhao, Amy Zeng, Michael Littman, et al.. Evidence Humans Provide When Explaining Data-Labeling Decisions. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.390-409, ⟨10.1007/978-3-030-29387-1_22⟩. ⟨hal-02553853⟩



Record views