Skip to Main content Skip to Navigation
Conference papers

Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors

Rituraj Kaushik 1 Timothée Anne 1 Jean-Baptiste Mouret 1
1 LARSEN - Lifelong Autonomy and interaction skills for Robots in a Sensing ENvironment
Inria Nancy - Grand Est, LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain where the dynamics of the robot can be significantly different from one another. In this paper, first, we show that when meta-training situations (the prior situations) have such diverse dynamics, using a single set of meta-trained parameters as a starting point still requires a large number of observations from the real system to learn a useful model of the dynamics. Second, we propose an algorithm called FAMLE that mitigates this limitation by meta-training several initial starting points (i.e., initial parameters) for training the model and allows robots to select the most suitable starting point to adapt the model to the current situation with only a few gradient steps. We compare FAMLE to MBRL, MBRL with a meta-trained model with MAML, and model-free policy search algorithm PPO for various simulated and real robotic tasks, and show that FAMLE allows robots to adapt to novel damages in significantly fewer time-steps than the baselines.
Complete list of metadata

Cited literature [32 references]  Display  Hide  Download

https://hal.inria.fr/hal-02909452
Contributor : Jean-Baptiste Mouret <>
Submitted on : Thursday, July 30, 2020 - 3:16:50 PM
Last modification on : Friday, July 31, 2020 - 3:42:29 AM
Long-term archiving on: : Tuesday, December 1, 2020 - 10:01:01 AM

File

famle-light.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02909452, version 1

Citation

Rituraj Kaushik, Timothée Anne, Jean-Baptiste Mouret. Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, Las Vegas, United States. ⟨hal-02909452⟩

Share

Metrics

Record views

80

Files downloads

108