https://hal.inria.fr/hal-00981135Loshchilov, IlyaIlyaLoshchilovLaboratory of Intelligent Systems (LIS) - LIS - Laboratory of Intelligent Systems - Laboratory of Intelligent Systems / EFPLA Computationally Efficient Limited Memory CMA-ES for Large Scale OptimizationHAL CCSD2014Evolution strategiesCMA-ESlarge scale optimizationCholesky update[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI]Ilya, Loshchilov2014-04-21 05:29:182014-10-13 15:43:252014-04-21 08:10:52enConference papershttps://hal.inria.fr/hal-00981135/documentapplication/pdf1We propose a computationally efficient limited memory Covariance Matrix Adaptation Evolution Strategy for large scale optimization, which we call the LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for numerical optimization of non-linear, non-convex optimization problems in continuous domain. Inspired by the limited memory BFGS method of Liu and Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a covariance matrix reproduced from $m$ direction vectors selected during the optimization process. The decomposition of the covariance matrix into Cholesky factors allows to reduce the time and memory complexity of the sampling to $O(mn)$, where $n$ is the number of decision variables. When $n$ is large (e.g., $n$ > 1000), even relatively small values of $m$ (e.g., $m=20,30$) are sufficient to efficiently solve fully non-separable problems and to reduce the overall run-time.