| HAL-Inria | Publications, software ... of Inria's scientists |
Conference papers
Using Global Statistics to Rank Retrieval Systems without Relevance Judgments
Abstract : How to reduce the amount of relevance judgments is an important issue in retrieval evaluation. In this paper, we propose a novel method using global statistics to rank retrieval systems without relevance judgments. In our method, a series of global statistics of a system, which indicate the percentage of its documents found by k out of all the N systems (k = 1, 2, ..., N), are selected, then a linear combination of the series of global statistics is utilized to fit the mean average precision (MAP) of the retrieval system. Optimal coefficients are obtained by linear regression. No human relevance judgments are required in the entire process. Compared with existing evaluation methods without relevance judgments, our method has two advantages. Firstly, it outperforms all early attempts. Secondly, it is adjustable for different effectiveness measurements, e.g. MAP, precision at n, and so forth.
Cited literature [12 references]
https://hal.inria.fr/hal-01055060
Contributor : Hal Ifip <>
Submitted on : Monday, August 11, 2014 - 12:58:55 PM
Last modification on : Thursday, March 5, 2020 - 5:43:12 PM
Long-term archiving on: : Wednesday, November 26, 2014 - 9:56:44 PM
Contributor : Hal Ifip <>
Submitted on : Monday, August 11, 2014 - 12:58:55 PM
Last modification on : Thursday, March 5, 2020 - 5:43:12 PM
Long-term archiving on: : Wednesday, November 26, 2014 - 9:56:44 PM
File
Using_Global_Statistics_to_Ran...
Files produced by the author(s)
Licence
Distributed under a Creative Commons Attribution 4.0 International License