Skip to Main content Skip to Navigation
Conference papers

Gamification for Word Sense Labeling

Abstract : Obtaining gold standard data for word sense disambiguation is important but costly. We show how it can be done using a " Game with a Purpose " (GWAP) called Wordrobe. This game consists of a large set of multiple-choice questions on word senses generated from the Groningen Meaning Bank. The players need to answer these questions, scoring points depending on the agreement with fellow players. The working assumption is that the right sense for a word can be determined by the answers given by the players. To evaluate our method, we gold-standard tagged a portion of the data that was also used in the GWAP. A comparison yielded promising results, ranging from a precision of 0.88 and recall of 0.83 for relative majority agreement, to a precision of 0.98 and recall of 0.35 for questions that were answered unanimously.
Complete list of metadata

Cited literature [11 references]  Display  Hide  Download

https://hal.inria.fr/hal-01342431
Contributor : Valerio Basile <>
Submitted on : Wednesday, July 6, 2016 - 9:02:37 AM
Last modification on : Monday, October 9, 2017 - 1:18:03 PM

File

W13-0215.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01342431, version 1

Collections

Citation

Noortje Venhuizen, Kilian Evang, Valerio Basile, Johan Bos. Gamification for Word Sense Labeling. Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Mar 2013, Potsdam, Germany. ⟨hal-01342431⟩

Share

Metrics

Record views

296

Files downloads

425