Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Memory-Augmented Reinforcement Learning for Image-Goal Navigation

Abstract : In this work, we address the problem of image-goal navigation in the context of visually-realistic 3D environments. This task involves navigating to a location indicated by a target image in a previously unseen environment. Earlier attempts, including RL-based and SLAM-based approaches, have either shown poor generalization performance, or are heavily-reliant on pose/depth sensors. We present a novel method that leverages a cross-episode memory to learn to navigate. We first train a state-embedding network in a self-supervised fashion, and then use it to embed previously-visited states into a memory. In order to avoid overfitting, we propose to use data augmentation on the RGB input during training. We validate our approach through extensive evaluations, showing that our data-augmented memory-based model establishes a new state of the art on the image-goal navigation task in the challenging Gibson dataset. We obtain this competitive performance from RGB input only, without access to additional sensors such as position or depth.
Document type :
Preprints, Working Papers, ...
Complete list of metadata
Contributor : Karteek Alahari Connect in order to contact the contributor
Submitted on : Thursday, January 14, 2021 - 9:38:11 PM
Last modification on : Tuesday, October 19, 2021 - 11:25:53 AM
Long-term archiving on: : Thursday, April 15, 2021 - 7:33:01 PM


Files produced by the author(s)


  • HAL Id : hal-03110875, version 1
  • ARXIV : 2101.05181



Lina Mezghani, Sainbayar Sukhbaatar, Thibaut Lavril, Oleksandr Maksymets, Dhruv Batra, et al.. Memory-Augmented Reinforcement Learning for Image-Goal Navigation. 2021. ⟨hal-03110875⟩



Les métriques sont temporairement indisponibles