Skip to Main content Skip to Navigation
Conference papers

Grounding Language to Autonomously-Acquired Skills via Goal Generation

Ahmed Akakzia 1 Cédric Colas 2 Pierre-Yves Oudeyer 2 Mohamed Chetouani 1, 3 Olivier Sigaud 1
2 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
3 PIROS - Perception, Interaction, Robotique sociales
ISIR - Institut des Systèmes Intelligents et de Robotique
Abstract : We are interested in the autonomous acquisition of repertoires of skills. Language-conditioned reinforcement learning (LC-RL) approaches are great tools in this quest, as they allow to express abstract goals as sets of constraints on the states. However, most LC-RL agents are not autonomous and cannot learn without external instructions and feedback. Besides, their direct language condition cannot account for the goal-directed behavior of pre-verbal infants and strongly limits the expression of behavioral diversity for a given language input. To resolve these issues, we propose a new conceptual approach to language-conditioned RL: the Language-Goal-Behavior architecture (LGB). LGB decouples skill learning and language grounding via an intermediate semantic representation of the world. To showcase the properties of LGB, we present a specific implementation called DECSTR. DECSTR is an intrinsically motivated learning agent endowed with an innate semantic representation describing spatial relations between physical objects. In a first stage (G -> B), it freely explores its environment and targets self-generated semantic configurations. In a second stage (L -> G), it trains a language-conditioned goal generator to generate semantic goals that match the constraints expressed in language-based inputs. We showcase the additional properties of LGB w.r.t. both an end-to-end LC-RL approach and a similar approach leveraging non-semantic, continuous intermediate representations. Intermediate semantic representations help satisfy language commands in a diversity of ways, enable strategy switching after a failure and facilitate language grounding.
Complete list of metadata

https://hal.inria.fr/hal-03121146
Contributor : Cédric Colas Connect in order to contact the contributor
Submitted on : Tuesday, January 26, 2021 - 10:49:29 AM
Last modification on : Monday, January 24, 2022 - 3:11:37 PM
Long-term archiving on: : Tuesday, April 27, 2021 - 6:27:07 PM

File

2006.07185.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03121146, version 1
  • ARXIV : 2006.07185

Citation

Ahmed Akakzia, Cédric Colas, Pierre-Yves Oudeyer, Mohamed Chetouani, Olivier Sigaud. Grounding Language to Autonomously-Acquired Skills via Goal Generation. ICLR 2021 - Ninth International Conference on Learning Representation, May 2021, Vienna / Virtual, Austria. ⟨hal-03121146⟩

Share

Metrics

Les métriques sont temporairement indisponibles