Skip to Main content Skip to Navigation
Journal articles

Challenge to test reproducibility of old computer code

Nicolas P. Rougier 1 Konrad Hinsen 2 
1 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : We question whether analytical tools such as Common Workflow Language, which aim to make computational methods “reproducible and shareable”, can stand the test of time (see Nature 573, 149–150; 2019). The long-term validity of computational results will not be testable if the original code cannot be run many years later. Considering the rapidity of transformations in operating systems and programming languages, it is hard to predict the lifetime reproducibility of a particular code. We have therefore organized the Ten Years Reproducibility Challenge (see Researchers are invited to test code reproducibility by trying to rerun a code created for a scientific paper they published more than ten years ago. The codes can address any scientific domain (statistical analysis, numerical simulation or data processing, for example) and be written in any language.The challenge closes in April 2020. Our hope is that the results will offer insights into long-term causes of non-reproducibility.
Document type :
Journal articles
Complete list of metadata

Cited literature [2 references]  Display  Hide  Download
Contributor : Nicolas P. Rougier Connect in order to contact the contributor
Submitted on : Wednesday, November 6, 2019 - 10:58:36 AM
Last modification on : Friday, July 8, 2022 - 10:06:58 AM
Long-term archiving on: : Saturday, February 8, 2020 - 4:03:39 AM


2019 - Putting reproducibility...
Files produced by the author(s)



Nicolas P. Rougier, Konrad Hinsen. Challenge to test reproducibility of old computer code. Nature, Nature Publishing Group, 2019, 574 (7780), pp.634. ⟨10.1038/d41586-019-03296-8⟩. ⟨hal-02340398⟩



Record views


Files downloads