Skip to Main content Skip to Navigation
Conference papers

Progressive growing of self-organized hierarchical representations for exploration

Mayalen Etcheverry 1 Pierre-Yves Oudeyer 1 Chris Reinke 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : Designing agent that can autonomously discover and learn a diversity of structures and skills in unknown changing environments is key for lifelong machine learning. A central challenge is how to learn incrementally representations in order to progressively build a map of the discovered structures and re-use it to further explore. To address this challenge, we identify and target several key functionalities. First, we aim to build lasting representations and avoid catastrophic forgetting throughout the exploration process. Secondly we aim to learn a diversity of representations allowing to discover a "diversity of diversity" of structures (and associated skills) in complex high-dimensional environments. Thirdly, we target representations that can structure the agent discoveries in a coarse-to-fine manner. Finally, we target the reuse of such representations to drive exploration toward an "interesting" type of diversity, for instance leveraging human guidance. Current approaches in state representation learning rely generally on monolithic architectures which do not enable all these functionalities. Therefore, we present a novel technique to progressively construct a Hierarchy of Observation Latent Models for Exploration Stratification, called HOLMES. This technique couples the use of a dynamic modular model architecture for representation learning with intrinsically-motivated goal exploration processes (IMGEPs). The paper shows results in the domain of automated discovery of diverse self-organized patterns, considering as testbed the experimental framework from Reinke et al. (2019).
Complete list of metadata

https://hal.inria.fr/hal-03122039
Contributor : Mayalen Etcheverry Connect in order to contact the contributor
Submitted on : Tuesday, January 26, 2021 - 6:17:31 PM
Last modification on : Friday, December 3, 2021 - 11:34:07 AM
Long-term archiving on: : Tuesday, April 27, 2021 - 7:32:10 PM

File

2005.06369.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03122039, version 1
  • ARXIV : 2005.06369

Collections

Citation

Mayalen Etcheverry, Pierre-Yves Oudeyer, Chris Reinke. Progressive growing of self-organized hierarchical representations for exploration. ICLR 2020 workshop: Beyond tabula rasa in Reinforcement Learning, Apr 2021, Addis Ababa / Virtual, Ethiopia. ⟨hal-03122039⟩

Share

Metrics

Les métriques sont temporairement indisponibles