3INGI - Département d'Ingénierie Informatique - UCL (Université catholique de Louvain Département d'ingénierie Informatique Department of Computing Science and Engineering 2, Place Sainte-Barbe B-1348 Louvain-la-Neuve - Belgium)
Abstract : A growing fraction of the papers published by CCR and at SIGCOMM-sponsored conferences include artifacts such as software or datasets. Besides CCR, these artifacts were rarely evaluated. During the last months of 2018, we organised two different Artifacts Evaluation Committees to which authors could submit the artifacts of their papers for evaluation. The first one evaluated the papers accepted by Conext’18 shortly after the TPC decision. It assigned ACM reproducibility badges to 12 different papers. The second one evaluated papers accepted by CCR and any SIGCOMM-sponsored conference. 28 papers received ACM reproducibility badges. We report on the results of a short survey among artifacts authors and reviewers and provide some suggestions for future artifacts evaluations.
Damien Saucez, Luigi Iannone, Olivier Bonaventure. Evaluating the artifacts of SIGCOMM papers. Computer Communication Review, Association for Computing Machinery, 2019, 49 (2), pp.44-47. ⟨10.1145/3336937.3336944⟩. ⟨hal-02433447⟩