HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Log-log Convergence for Noisy Optimization

Sandra Cecilia Astete-Morales 1, 2 Jialin Liu 1, 2 Olivier Teytaud 2, 1
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : We consider noisy optimization problems, without the as-sumption of variance vanishing in the neighborhood of the optimum. We show mathematically that simple rules with exponential number of re-samplings lead to a log-log convergence rate. In particular, in this case the log of the distance to the optimum is linear on the log of the num-ber of resamplings. As well as with number of resamplings polynomial in the inverse step-size. We show empirically that this convergence rate is obtained also with polynomial number of resamplings. In this polyno-mial resampling setting, using classical evolution strategies and an ad hoc choice of the number of resamplings, we seemingly get the same rate as those obtained with specific Estimation of Distribution Algorithms de-signed for noisy setting. We also experiment non-adaptive polynomial re-samplings. Compared to the state of the art, our results provide (i) proofs of log-log convergence for evolution strategies (which were not covered by existing results) in the case of objective functions with quadratic expec-tations and constant noise, (ii) log-log rates also for objective functions with expectation E[f (x)] = ||x − x * || p , where x * represents the optimum (iii) experiments with different parametrizations than those considered in the proof. These results propose some simple revaluation schemes. This paper extends [1].
Document type :
Conference papers
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download

Contributor : Olivier Teytaud Connect in order to contact the contributor
Submitted on : Monday, October 19, 2015 - 10:06:15 AM
Last modification on : Thursday, July 8, 2021 - 3:50:29 AM
Long-term archiving on: : Thursday, April 27, 2017 - 6:02:55 AM


Files produced by the author(s)



Sandra Cecilia Astete-Morales, Jialin Liu, Olivier Teytaud. Log-log Convergence for Noisy Optimization. Evolutionary Algorithms 2013, Oct 2013, Bordeaux, France. pp.16 - 28, ⟨10.1007/978-3-319-11683-9_2⟩. ⟨hal-01107772v2⟩



Record views


Files downloads