Reproducibility of Experiments in Recommender Systems Evaluation

Abstract : Recommender systems evaluation is usually based on predictive accuracy metrics with better scores meaning recommendations of higher quality. However, the comparison of results is becoming increasingly difficult, since there are different recommendation frameworks and different settings in the design and implementation of the experiments. Furthermore, there might be minor differences on algorithm implementation among the different frameworks. In this paper, we compare well known recommendation algorithms, using the same dataset, metrics and overall settings, the results of which point to result differences across frameworks with the exact same settings. Hence, we propose the use of standards that should be followed as guidelines to ensure the replication of experiments and the reproducibility of the results.
Document type :
Conference papers
Lazaros Iliadis; Ilias Maglogiannis; Vassilis Plagianakos. 14th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), May 2018, Rhodes, Greece. Springer International Publishing, IFIP Advances in Information and Communication Technology, AICT-519, pp.401-409, 2018, Artificial Intelligence Applications and Innovations. 〈10.1007/978-3-319-92007-8_34〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01821035
Contributor : Hal Ifip <>
Submitted on : Friday, June 22, 2018 - 11:44:17 AM
Last modification on : Friday, June 22, 2018 - 12:00:56 PM
Document(s) archivé(s) le : Tuesday, September 25, 2018 - 7:46:30 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2021-01-01

Please log in to resquest access to the document

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Nikolaos Polatidis, Stelios Kapetanakis, Elias Pimenidis, Konstantinos Kosmidis. Reproducibility of Experiments in Recommender Systems Evaluation. Lazaros Iliadis; Ilias Maglogiannis; Vassilis Plagianakos. 14th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), May 2018, Rhodes, Greece. Springer International Publishing, IFIP Advances in Information and Communication Technology, AICT-519, pp.401-409, 2018, Artificial Intelligence Applications and Innovations. 〈10.1007/978-3-319-92007-8_34〉. 〈hal-01821035〉

Share

Metrics

Record views

20