Statistical Performance Metrics for Use with Imprecise Ground-Truth

Abstract : This paper addresses performance evaluation in the presence of imprecise ground-truth. Indeed, the most common assumption when performing benchmarking measures is that the reference data is awless. In previous work, we have shown that this assumption cannot be taken for granted, and that, in the case of perceptual interpretation problems it is most certainly always wrong but for the most trivial cases. We are presenting a statistical test that will allow measuring the con-dence one can have in the results of a benchmarking test ranking multiple algorithms. More specically, we can express the probability of the ranking not being respected in the presence of a given level of errors in the ground truth data.
Complete list of metadatas

Cited literature [8 references]  Display  Hide  Download

https://hal.inria.fr/hal-01401034
Contributor : Bart Lamiroy <>
Submitted on : Tuesday, November 22, 2016 - 6:13:42 PM
Last modification on : Thursday, October 17, 2019 - 12:36:54 PM
Long-term archiving on : Tuesday, March 21, 2017 - 9:09:00 AM

File

9657_35.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution - NonCommercial 4.0 International License

Identifiers

Citation

Bart Lamiroy, Pascal Pierrot. Statistical Performance Metrics for Use with Imprecise Ground-Truth. Graphics Recognition. Current Trends and Challenges: 11th International Workshop on Graphics Recognition, GREC 2015, Aug 2015, Nancy, France. ⟨10.1007/978-3-319-52159-6_3⟩. ⟨hal-01401034⟩

Share

Metrics

Record views

369

Files downloads

190