M. Andrychowicz, Hindsight experience replay, Proc. of NIPS, 2017.

M. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton et al., Unifying count-based exploration and intrinsic motivation, Proc. of NIPS, 2016.

L. Breiman, Random forests. Machine learning, vol.45, pp.5-32, 2001.

K. Chatzilygeroudis and J. Mouret, Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics, Proc. of ICRA, 2018.
DOI : 10.1109/icra.2018.8461083
URL : https://hal.archives-ouvertes.fr/hal-01768285

K. Chatzilygeroudis, R. Rama, R. Kaushik, and D. Goepp, Vassilis Vassiliades, and Jean-Baptiste Mouret. Black-Box Data-efficient Policy Search for Robotics, Proc. of IROS, 2017.

C. Colas, O. Sigaud, and P. Oudeyer, GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms, Proc. of ICML, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01840576

A. Cully, K. Chatzilygeroudis, F. Allocati, and J. Mouret, Limbo: A Flexible Highperformance Library for Gaussian Processes modeling and Data-Efficient Optimization, The Journal of Open Source Software, vol.3, issue.26, p.545, 2018.
DOI : 10.21105/joss.00545
URL : https://hal.archives-ouvertes.fr/hal-01884299

A. Cully and Y. Demiris, Quality and diversity optimization: A unifying modular framework, IEEE Trans. on Evolutionary Computation, vol.22, issue.2, pp.245-259, 2018.
DOI : 10.1109/tevc.2017.2704781
URL : https://doi.org/10.1109/tevc.2017.2704781

K. Deb and D. Kalyanmoy, Multi-Objective Optimization Using Evolutionary Algorithms, 2001.

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans. on Evolutionary Computation, vol.6, issue.2, pp.182-197, 2002.
DOI : 10.1109/4235.996017
URL : http://work.caltech.edu/amrit/papers/nsga2.ps.gz

M. Peter-deisenroth, D. Fox, and C. E. Rasmussen, Gaussian processes for dataefficient learning in robotics and control, IEEE Trans. Pattern Anal. Mach. Intell, vol.37, issue.2, pp.408-423, 2015.

M. Peter-deisenroth, G. Neumann, and J. Peters, A survey on policy search for robotics, Foundations and Trends in Robotics, vol.2, issue.1, pp.1-142, 2013.

S. Doncieux and J. Mouret, Beyond black-box optimization: a review of selective pressures for evolutionary robotics, Evolutionary Intelligence, vol.7, issue.2, pp.71-93, 2014.
URL : https://hal.archives-ouvertes.fr/hal-01150254

C. Florensa, D. Held, M. Wulfmeier, and P. Abbeel, Reverse curriculum generation for reinforcement learning, Conference on Robot Learning, 2017.

S. Forestier, Y. Mollard, and P. Oudeyer, Intrinsically motivated goal exploration processes with automatic curriculum learning, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01651233

S. Forestier and P. Oudeyer, Curiosity-driven development of tool use precursors: a computational model, Proc. of COGSCI, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01354013

. David-e-goldberg, Simple genetic algorithms and the minimal, deceptive problem. Genetic algorithms and simulated annealing, pp.74-88, 1987.

J. Gottlieb and P. Oudeyer, Information-seeking, curiosity, and attention: computational and neural mechanisms, Trends in Cognitive Sciences, vol.17, issue.11, pp.585-593, 2013.
DOI : 10.1016/j.tics.2013.09.001
URL : https://hal.archives-ouvertes.fr/hal-00913646

N. Hansen, The CMA Evolution Strategy: A Comparing Review, 2006.
DOI : 10.1007/11007937_4
URL : http://www.bionik.tu-berlin.de/user/niko/hansenedacomparing.pdf

N. Heess, Emergence of locomotion behaviours in rich environments, 2017.

R. Houthooft, X. Chen, Y. Duan, J. Schulman, F. D. Turck et al., Vime: Variational information maximizing exploration, Proc. of NIPS, 2016.

L. Pack-kaelbling, A. Michael-l-littman, and . Moore, Reinforcement learning: A survey, Journal of artificial intelligence research, vol.4, pp.237-285, 1996.

M. Kearns and S. Singh, Near-optimal reinforcement learning in polynomial time, Machine learning, vol.49, issue.2-3, pp.209-232, 2002.

J. Kober and J. Peters, Policy search for motor primitives in robotics, Machine Learning, vol.84, pp.171-203, 2011.

J. Lehman, . Kenneth, and . Stanley, Abandoning objectives: Evolution through the search for novelty alone, Evolutionary computation, vol.19, issue.2, pp.189-223, 2011.

T. Paul-lillicrap, Continuous control with deep reinforcement learning, Proc. of ICLR, 2016.

M. Lopes, T. Lang, M. Toussaint, and P. Oudeyer, Exploration in modelbased reinforcement learning by empirically estimating learning progress, Proc. of NIPS, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00755248

V. Mnih, Human-level control through deep reinforcement learning, Nature, vol.518, issue.7540, pp.529-533, 2015.

S. Mohamed and D. Jimenez-rezende, Variational information maximisation for intrinsically motivated reinforcement learning, Proc. of NIPS, 2015.

T. Mihai-moldovan, S. Levine, P. Michael-i-jordan, and . Abbeel, Optimismdriven exploration for nonlinear systems, Proc. of ICRA, 2015.

J. Mouret, Novelty-based Multiobjectivization, New Horizons in Evolutionary Robotics, pp.139-154, 2011.
DOI : 10.1007/978-3-642-18272-3_10
URL : https://hal.archives-ouvertes.fr/hal-01300711

J. Mouret and J. Clune, Illuminating search spaces by mapping elites, 2015.

J. Mouret and S. Doncieux, Sferes v2: Evolvin'in the multi-core world, Proc. of CEC, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00687633

P. Oudeyer, F. Kaplan, and V. V. Hafner, Intrinsic motivation systems for autonomous mental development, IEEE Trans. on Evolutionary Computation, vol.11, issue.2, pp.265-286, 2007.

P. Oudeyer, F. Kaplan, V. Verena, A. Hafner, and . Whyte, The playground experiment: Task-independent development of a curious robot, Proc. of the AAAI Spring Symposium on Developmental Robotics, pp.42-47, 2005.

D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, Curiosity-driven exploration by self-supervised prediction, Proc. of ICML, 2017.

C. E. Rasmussen and C. K. Williams, Gaussian processes for machine learning, 2006.

M. Riedmiller, , 2018.

J. Schmidhuber, A Possibility for Implementing Curiosity and Boredom in Modelbuilding Neural Controllers, Proc. of SAB, 1990.

J. Schulman, S. Levine, P. Moritz, P. Michael-i-jordan, and . Abbeel, Trust region policy optimization, Proc. of ICML, 2015.

D. Silver, Mastering the game of go without human knowledge, Nature, vol.550, issue.7676, p.354, 2017.

C. Xie, S. Patil, T. Moldovan, S. Levine, and P. Abbeel, Model-based reinforcement learning with parametrized physical models and optimism-driven exploration, Proc. of ICRA, 2016.

A. Auger and N. Hansen, A restart CMA evolution strategy with increasing population size, Congress on Evolutionary Computation, 2005.

M. Blum and . Martin-a-riedmiller, Optimization of Gaussian process hyperparameters using Rprop, Proc. of ESANN, 2013.

A. Cully, K. Chatzilygeroudis, F. Allocati, and J. Mouret, Limbo: A Flexible Highperformance Library for Gaussian Processes modeling and Data-Efficient Optimization, The Journal of Open Source Software, vol.3, issue.26, p.545, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01884299

K. Deb and H. Beyer, Self-adaptive genetic algorithms with simulated binary crossover. Secretary of the SFB 531, 1999.
DOI : 10.1162/106365601750190406
URL : https://eldorado.tu-dortmund.de/bitstream/2003/5370/1/ci61.pdf

K. Deb and D. Kalyanmoy, Multi-Objective Optimization Using Evolutionary Algorithms, 2001.

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans. on Evolutionary Computation, vol.6, issue.2, pp.182-197, 2002.

J. Lee, X. Michael, S. Grey, T. Ha, S. Kunz et al., DART: Dynamic animation and robotics toolkit, The Journal of Open Source Software, 2018.
DOI : 10.21105/joss.00500
URL : https://www.theoj.org/joss-papers/joss.00500/10.21105.joss.00500.pdf

J. Lehman, J. Chen, J. Clune, and K. Stanley, Es is more than just a traditional finite-difference approximator, 2017.

I. Loshchilov, CMA-ES with restarts for solving CEC 2013 benchmark problems, Congress on Evolutionary Computation, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00823880

J. Mouret and S. Doncieux, Sferes v2: Evolvin'in the multi-core world, Proc. of CEC, 2010.
URL : https://hal.archives-ouvertes.fr/hal-00687633

F. P. Such, V. Madhavan, E. Conti, J. Lehman, O. Kenneth et al., Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning, 2017.

C. E. Rasmussen and C. K. Williams, Gaussian processes for machine learning, 2006.

M. Riedmiller and H. Braun, Rprop-a fast adaptive learning algorithm, Proc. of ISCIS VII, 1992.

T. Salimans, J. Ho, X. Chen, S. Sidor, and I. Sutskever, Evolution strategies as a scalable alternative to reinforcement learning, 2017.