Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning

Grgur Kovač 1 Adrien Laversanne-Finot 1 Pierre-Yves Oudeyer 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : Autonomous agents using novelty based goal exploration are often efficient in environments that require exploration. However, they get attracted to various forms of distracting unlearnable regions. To solve this problem, absolute learning progress (ALP) has been used in reinforcement learning agents with predefined goal features and access to expert knowledge. This work extends those concepts to unsupervised image-based goal exploration. We present the GRIMGEP framework: it provides a learned robust goal sampling prior that can be used on top of current state-of-the-art novelty seeking goal exploration approaches, enabling them to ignore noisy distracting regions while searching for novelty in the learnable regions. It clusters the goal space and estimates ALP for each cluster. These ALP estimates can then be used to detect the distracting regions, and build a prior that enables further goal sampling mechanisms to ignore them. We construct an image based environment with distractors, on which we show that wrapping current state-of-the-art goal exploration algorithms with our framework allows them to concentrate on interesting regions of the environment and drastically improve performances. The source code is available at https://sites.google.com/view/grimgep.
Complete list of metadatas

https://hal.inria.fr/hal-03121352
Contributor : Grgur Kovac <>
Submitted on : Tuesday, January 26, 2021 - 12:37:42 PM
Last modification on : Friday, January 29, 2021 - 3:39:05 PM

File

GRIMGEP.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03121352, version 1
  • ARXIV : 2008.04388

Collections

Citation

Grgur Kovač, Adrien Laversanne-Finot, Pierre-Yves Oudeyer. GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning. 2021. ⟨hal-03121352⟩

Share

Metrics

Record views

39

Files downloads

91