Skip to Main content Skip to Navigation
Conference papers

GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms

Cédric Colas 1 Olivier Sigaud 2, 1 Pierre-Yves Oudeyer 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : In continuous action domains, standard deep reinforcement learning algorithms like DDPG suffer from inefficient exploration when facing sparse or deceptive reward problems. Conversely, evolutionary and developmental methods focusing on exploration like Novelty Search, Quality-Diversity or Goal Exploration Processes explore more robustly but are less efficient at fine-tuning policies using gradient-descent. In this paper, we present the GEP-PG approach, taking the best of both worlds by sequentially combining a Goal Exploration Process and two variants of DDPG. We study the learning performance of these components and their combination on a low dimensional deceptive reward problem and on the larger Half-Cheetah benchmark. We show that DDPG fails on the former and that GEP-PG improves over the best DDPG variant in both environments. Supplementary videos and discussion can be found at frama.link/gep_pg, the code at github.com/flowersteam/geppg.
Complete list of metadatas

Cited literature [47 references]  Display  Hide  Download

https://hal.inria.fr/hal-01890151
Contributor : Cédric Colas <>
Submitted on : Monday, October 8, 2018 - 1:36:03 PM
Last modification on : Wednesday, July 3, 2019 - 10:48:04 AM
Long-term archiving on: : Wednesday, January 9, 2019 - 2:26:02 PM

File

1802.05054.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01890151, version 1

Citation

Cédric Colas, Olivier Sigaud, Pierre-Yves Oudeyer. GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms. International Conference on Machine Learning (ICML), Jul 2018, Stockholm, Sweden. ⟨hal-01890151⟩

Share

Metrics

Record views

167

Files downloads

90