Using Global Statistics to Rank Retrieval Systems without Relevance Judgments

Abstract : How to reduce the amount of relevance judgments is an important issue in retrieval evaluation. In this paper, we propose a novel method using global statistics to rank retrieval systems without relevance judgments. In our method, a series of global statistics of a system, which indicate the percentage of its documents found by k out of all the N systems (k = 1, 2, ..., N), are selected, then a linear combination of the series of global statistics is utilized to fit the mean average precision (MAP) of the retrieval system. Optimal coefficients are obtained by linear regression. No human relevance judgments are required in the entire process. Compared with existing evaluation methods without relevance judgments, our method has two advantages. Firstly, it outperforms all early attempts. Secondly, it is adjustable for different effectiveness measurements, e.g. MAP, precision at n, and so forth.
Type de document :
Communication dans un congrès
Zhongzhi Shi; Sunil Vadera; Agnar Aamodt; David Leake. 6th IFIP TC 12 International Conference on Intelligent Information Processing (IIP), Oct 2010, Manchester, United Kingdom. Springer, IFIP Advances in Information and Communication Technology, AICT-340, pp.183-192, 2010, Intelligent Information Processing V. 〈10.1007/978-3-642-16327-2_24〉
Liste complète des métadonnées

Littérature citée [12 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01055060
Contributeur : Hal Ifip <>
Soumis le : lundi 11 août 2014 - 12:58:55
Dernière modification le : vendredi 11 août 2017 - 14:50:38
Document(s) archivé(s) le : mercredi 26 novembre 2014 - 21:56:44

Fichier

Using_Global_Statistics_to_Ran...
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Zhiwei Shi, Bin Wang, Peng Li, Zhongzhi Shi. Using Global Statistics to Rank Retrieval Systems without Relevance Judgments. Zhongzhi Shi; Sunil Vadera; Agnar Aamodt; David Leake. 6th IFIP TC 12 International Conference on Intelligent Information Processing (IIP), Oct 2010, Manchester, United Kingdom. Springer, IFIP Advances in Information and Communication Technology, AICT-340, pp.183-192, 2010, Intelligent Information Processing V. 〈10.1007/978-3-642-16327-2_24〉. 〈hal-01055060〉

Partager

Métriques

Consultations de la notice

241

Téléchargements de fichiers

108