Benchmarking Numerical Multiobjective Optimizers Revisited

Dimo Brockhoff 1, * Thanh-Do Tran 1 Nikolaus Hansen 2
* Auteur correspondant
1 DOLPHIN - Parallel Cooperative Multi-criteria Optimization
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
2 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
Abstract : Algorithm benchmarking plays a vital role in designing new optimization algorithms and in recommending efficient and robust algorithms for practical purposes. So far, two main approaches have been used to compare algorithms in the evolutionary multiobjective optimization (EMO) field: (i) displaying empirical attainment functions and (ii) reporting statistics on quality indicator values. Most of the time, EMO benchmarking studies compare algorithms for fixed and often arbitrary budgets of function evaluations although the algorithms are anytime optimizers. Instead, we propose to transfer and adapt standard benchmarking techniques from the single-objective optimization and classical derivative-free optimization community to the field of EMO. Reporting target-based runlengths allows to compare algorithms with varying numbers of function evaluations quantitatively. Displaying data profiles can aggregate performance information over different test functions, problem difficulties, and quality indicators. We apply this approach to compare three common algorithms on a new test function suite derived from the well-known single-objective BBOB functions. The focus thereby lies less on gaining insights into the algorithms but more on showcasing the concepts and on what can be gained over current benchmarking approaches.
Type de document :
Communication dans un congrès
A.I. Esparcia and S. Silva. Genetic and Evolutionary Computation Conference (GECCO 2015), Jul 2015, Madrid, Spain. pp.639-646, <10.1145/2739480.2754777>
Liste complète des métadonnées
Contributeur : Dimo Brockhoff <>
Soumis le : mercredi 29 avril 2015 - 00:21:02
Dernière modification le : vendredi 17 février 2017 - 16:10:41
Document(s) archivé(s) le : lundi 14 septembre 2015 - 14:57:12


Fichiers produits par l'(les) auteur(s)



Dimo Brockhoff, Thanh-Do Tran, Nikolaus Hansen. Benchmarking Numerical Multiobjective Optimizers Revisited. A.I. Esparcia and S. Silva. Genetic and Evolutionary Computation Conference (GECCO 2015), Jul 2015, Madrid, Spain. pp.639-646, <10.1145/2739480.2754777>. <hal-01146741>



Consultations de
la notice


Téléchargements du document