Maximum Likelihood-based Online Adaptation of Hyper-parameters in CMA-ES

Ilya Loshchilov 1 Marc Schoenauer 2, 3 Michèle Sebag 4 Nikolaus Hansen 3
1 Laboratory of Intelligent Systems (LIS)
LIS - Laboratory of Intelligent Systems
3 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
4 Laboratoire de Recherche en Informatique
LRI - Laboratoire de Recherche en Informatique
Abstract : The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is widely accepted as a robust derivative-free continuous optimization algorithm for non-linear and non-convex optimization problems. CMA-ES is well known to be almost parameterless, meaning that only one hyper-parameter, the population size, is proposed to be tuned by the user. In this paper, we propose a principled approach called self-CMA-ES to achieve the online adaptation of CMA-ES hyper-parameters in order to improve its overall performance. Experimental results show that for larger-than-default population size, the default settings of hyper-parameters of CMA-ES are far from being optimal, and that self-CMA-ES allows for dynamically approaching optimal settings.
Document type :
Conference papers
Th. Bartz-Beielstein and J. Branke and B. Filipič and J. Smith. 13th International Conference on Parallel Problem Solving from Nature (PPSN 2014), Sep 2014, Ljubljana, Slovenia. Springer Verlag, 8672, pp.70-79, 2014, Lecture Notes in Computer Science
Liste complète des métadonnées

Cited literature [16 references]  Display  Hide  Download

https://hal.inria.fr/hal-01003504
Contributor : Loshchilov Ilya <>
Submitted on : Wednesday, June 11, 2014 - 11:47:12 AM
Last modification on : Thursday, July 20, 2017 - 9:27:00 AM
Document(s) archivé(s) le : Thursday, September 11, 2014 - 11:50:27 AM

Files

selfCMA.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01003504, version 2
  • ARXIV : 1406.2623

Citation

Ilya Loshchilov, Marc Schoenauer, Michèle Sebag, Nikolaus Hansen. Maximum Likelihood-based Online Adaptation of Hyper-parameters in CMA-ES. Th. Bartz-Beielstein and J. Branke and B. Filipič and J. Smith. 13th International Conference on Parallel Problem Solving from Nature (PPSN 2014), Sep 2014, Ljubljana, Slovenia. Springer Verlag, 8672, pp.70-79, 2014, Lecture Notes in Computer Science. 〈hal-01003504v2〉

Share

Metrics

Record views

490

Document downloads

290