Skip to Main content Skip to Navigation
Conference papers

Evidence Humans Provide When Explaining Data-Labeling Decisions

Abstract : Because machine learning would benefit from reduced data requirements, some prior work has proposed using humans not just to label data, but also to explain those labels. To characterize the evidence humans might want to provide, we conducted a user study and a data experiment. In the user study, 75 participants provided classification labels for 20 photos, justifying those labels with free-text explanations. Explanations frequently referenced concepts (objects and attributes) in the image, yet 26% of explanations invoked concepts not in the image. Boolean logic was common in implicit form, but was rarely explicit. In a follow-up experiment on the Visual Genome dataset, we found that some concepts could be partially defined through their relationship to frequently co-occurring concepts, rather than only through labeling.
Document type :
Conference papers
Complete list of metadata

Cited literature [21 references]  Display  Hide  Download

https://hal.inria.fr/hal-02553853
Contributor : Hal Ifip <>
Submitted on : Friday, April 24, 2020 - 6:08:41 PM
Last modification on : Friday, April 24, 2020 - 6:39:47 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2022-01-01

Please log in to resquest access to the document

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Judah Newman, Bowen Wang, Valerie Zhao, Amy Zeng, Michael Littman, et al.. Evidence Humans Provide When Explaining Data-Labeling Decisions. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.390-409, ⟨10.1007/978-3-030-29387-1_22⟩. ⟨hal-02553853⟩

Share

Metrics

Record views

27