Ontology matching benchmarks: generation and evaluation

Maria Rosoiu 1 Cássia Trojahn dos Santos 1 Jérôme Euzenat 1
1 EXMO - Computer mediated exchange of structured knowledge
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
Abstract : The OAEI Benchmark data set has been used as a main reference to evaluate and compare matching systems. It requires matching an ontology with systematically modified versions of itself. However, it has two main drawbacks: it has not varied since 2004 and it has become a relatively easy task for matchers. In this paper, we present the design of a modular test generator that overcomes these drawbacks. Using this generator, we have reproduced Benchmark both with the original seed ontology and with other ontologies. Evaluating different matchers on these generated tests, we have observed that (a) the difficulties encountered by a matcher at a test are preserved across the seed ontology, (b) contrary to our expectations, we found no systematic positive bias towards the original data set which has been available for developers to test their systems, and (c) the generated data sets have consistent results across matchers and across seed ontologies. However, the discriminant power of the generated tests is still too low and more tests would be necessary to draw definitive conclusions.
Type de document :
Communication dans un congrès
Proc. 6th ISWC workshop on ontology matching (OM), Oct 2011, Bonn, Germany. No commercial editor., pp.73-84, 2011
Liste complète des métadonnées

Littérature citée [11 références]  Voir  Masquer  Télécharger

Contributeur : Jérôme Euzenat <>
Soumis le : vendredi 25 janvier 2013 - 11:31:35
Dernière modification le : vendredi 12 octobre 2018 - 01:18:13
Document(s) archivé(s) le : vendredi 26 avril 2013 - 03:55:44


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-00781026, version 1



Maria Rosoiu, Cássia Trojahn dos Santos, Jérôme Euzenat. Ontology matching benchmarks: generation and evaluation. Proc. 6th ISWC workshop on ontology matching (OM), Oct 2011, Bonn, Germany. No commercial editor., pp.73-84, 2011. 〈hal-00781026〉



Consultations de la notice


Téléchargements de fichiers