Gamification for Word Sense Labeling

Abstract : Obtaining gold standard data for word sense disambiguation is important but costly. We show how it can be done using a " Game with a Purpose " (GWAP) called Wordrobe. This game consists of a large set of multiple-choice questions on word senses generated from the Groningen Meaning Bank. The players need to answer these questions, scoring points depending on the agreement with fellow players. The working assumption is that the right sense for a word can be determined by the answers given by the players. To evaluate our method, we gold-standard tagged a portion of the data that was also used in the GWAP. A comparison yielded promising results, ranging from a precision of 0.88 and recall of 0.83 for relative majority agreement, to a precision of 0.98 and recall of 0.35 for questions that were answered unanimously.
Type de document :
Communication dans un congrès
Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Mar 2013, Potsdam, Germany
Liste complète des métadonnées

Littérature citée [11 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01342431
Contributeur : Valerio Basile <>
Soumis le : mercredi 6 juillet 2016 - 09:02:37
Dernière modification le : lundi 9 octobre 2017 - 13:18:03

Fichier

W13-0215.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01342431, version 1

Collections

Citation

Noortje Venhuizen, Kilian Evang, Valerio Basile, Johan Bos. Gamification for Word Sense Labeling. Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Mar 2013, Potsdam, Germany. 〈hal-01342431〉

Partager

Métriques

Consultations de la notice

101

Téléchargements de fichiers

83