Benchmarking Numerical Multiobjective Optimizers Revisited

Dimo Brockhoff 1, * Thanh-Do Tran 1 Nikolaus Hansen 2
* Corresponding author
1 DOLPHIN - Parallel Cooperative Multi-criteria Optimization
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
2 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : Algorithm benchmarking plays a vital role in designing new optimization algorithms and in recommending efficient and robust algorithms for practical purposes. So far, two main approaches have been used to compare algorithms in the evolutionary multiobjective optimization (EMO) field: (i) displaying empirical attainment functions and (ii) reporting statistics on quality indicator values. Most of the time, EMO benchmarking studies compare algorithms for fixed and often arbitrary budgets of function evaluations although the algorithms are anytime optimizers. Instead, we propose to transfer and adapt standard benchmarking techniques from the single-objective optimization and classical derivative-free optimization community to the field of EMO. Reporting target-based runlengths allows to compare algorithms with varying numbers of function evaluations quantitatively. Displaying data profiles can aggregate performance information over different test functions, problem difficulties, and quality indicators. We apply this approach to compare three common algorithms on a new test function suite derived from the well-known single-objective BBOB functions. The focus thereby lies less on gaining insights into the algorithms but more on showcasing the concepts and on what can be gained over current benchmarking approaches.
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download

https://hal.inria.fr/hal-01146741
Contributor : Dimo Brockhoff <>
Submitted on : Wednesday, April 29, 2015 - 12:21:02 AM
Last modification on : Friday, March 22, 2019 - 1:34:56 AM
Long-term archiving on : Monday, September 14, 2015 - 2:57:12 PM

File

mobmk-authorversion.pdf
Files produced by the author(s)

Identifiers

Citation

Dimo Brockhoff, Thanh-Do Tran, Nikolaus Hansen. Benchmarking Numerical Multiobjective Optimizers Revisited. Genetic and Evolutionary Computation Conference (GECCO 2015), Jul 2015, Madrid, Spain. pp.639-646, ⟨10.1145/2739480.2754777⟩. ⟨hal-01146741⟩

Share

Metrics

Record views

876

Files downloads

769