inria-00099856, version 1
Online Evaluation of Coreference Resolution
Andrei Popescu-Belis aLoïs Rigouste b, 1Susanne Salmon-Alt c, 2Laurent Romary
d, 3
4th International Conference on Language Resources and Evaluation - LREC'04 (2004) 4 p
Résumé : This paper presents the design of an online evaluation service for coreference resolution in texts. We argue that coreference, as an equivalence relation between referring expressions (RE) in texts, should be properly distinguished from anaphora and has therefore to be evaluated separately. The annotation model for coreference is based on links between REs. The program presented in this article compares two such annotations, which may be the output of coreference resolution tools or of human judgement. In order to evaluate the agreement between the two annotations, the evaluator first converts the input annotation format into a pivot format, then abstracts equivalence classes from the links and provides five scores representing in different ways the similarity between the two partitions: MUC, B3, Kappa, Core-discourse-entity, and Mutual-information. Although we consider that the identification of REs (i.e. the elements of the partition) should not be part of coreference resolution properly speaking, we propose several solutions for the frequent case when the input files do not agree on the elements of the text to consider as REs.
- a – ISSCO/TIM/ETI
- b – ENST
- c – CNRS
- d – INRIA
- 1 : Département Innovation Pédagogique
- Institut Mines-Télécom – Télécom ParisTech
- 2 : Analyse et Traitement Informatique de la Langue Française (ATILF)
- CNRS : UMR7118 – Université Henri Poincaré - Nancy I – Université Nancy II
- 3 : LANGUE ET DIALOGUE (INRIA Lorraine - LORIA)
- INRIA – CNRS : UMR7503 – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)
- Domaine : Informatique/Autre
- Référence interne : A04-R-554 || popescu-belis04a
- Commentaire : Colloque avec actes et comité de lecture. internationale.
- inria-00099856, version 1
- http://hal.inria.fr/inria-00099856
- oai:hal.inria.fr:inria-00099856
- Contributeur : Laurent Romary
- Soumis le : Mardi 13 Janvier 2009, 11:18:10
- Dernière modification le : Mardi 13 Janvier 2009, 11:18:10






Documents associés

Exporter