Evaluating Commonsense Knowledge with a Computer Game

Abstract : Collecting commonsense knowledge from freely available text can reduce the cost and effort of creating large knowledge bases. For the acquired knowledge to be useful, we must ensure that it is correct, and that it carries information about its relevance and about the context in which it can be considered commonsense. In this paper, we design, and evaluate an online game that classifies, using the input from players, text extracted from the web as either commonsense knowledge, domain-specific knowledge, or nonsense. A continuous scale is defined to classify the knowledge as nonsense or commonsense and it is later used during the evaluation of the data to identify which knowledge is reliable and which one needs further qualification. When comparing our results to other similar knowledge acquisition systems, our game performs better with respect to coverage, redundancy, and reliability of the commonsense acquired.
Document type :
Conference papers
Complete list of metadatas

Cited literature [12 references]  Display  Hide  Download

https://hal.inria.fr/hal-01590550
Contributor : Hal Ifip <>
Submitted on : Tuesday, September 19, 2017 - 4:59:06 PM
Last modification on : Thursday, August 1, 2019 - 2:12:40 PM

File

978-3-642-23774-4_28_Chapter.p...
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Juan Mancilla-Caceres, Eyal Amir. Evaluating Commonsense Knowledge with a Computer Game. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.348-355, ⟨10.1007/978-3-642-23774-4_28⟩. ⟨hal-01590550⟩

Share

Metrics

Record views

384

Files downloads

206