Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics

Konstantinos Chatzilygeroudis 1 Jean-Baptiste Mouret 1
1 LARSEN - Lifelong Autonomy and interaction skills for Robots in a Sensing ENvironment
Inria Nancy - Grand Est, LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms , which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recently introduced Black-DROPS algorithm exploits a black-box optimization algorithm to achieve both high data-efficiency and good computation times when several cores are used; nevertheless, like all model-based policy search approaches, Black-DROPS does not scale to high dimensional state/action spaces. In this paper, we introduce a new model learning procedure in Black-DROPS that leverages parameterized black-box priors to (1) scale up to high-dimensional systems, and (2) be robust to large inaccuracies of the prior information. We demonstrate the effectiveness of our approach with the " pendubot " swing-up task in simulation and with a physical hexapod robot (48D state space, 18D action space) that has to walk forward as fast as possible. The results show that our new algorithm is more data-efficient than previous model-based policy search algorithms (with and without priors) and that it can allow a physical 6-legged robot to learn new gaits in only 16 to 30 seconds of interaction time.
Complete list of metadatas

Cited literature [50 references]  Display  Hide  Download

https://hal.inria.fr/hal-01768285
Contributor : Jean-Baptiste Mouret <>
Submitted on : Tuesday, April 17, 2018 - 10:17:27 AM
Last modification on : Tuesday, December 18, 2018 - 4:40:22 PM

File

1709.06917.pdf
Files produced by the author(s)

Identifiers

Citation

Konstantinos Chatzilygeroudis, Jean-Baptiste Mouret. Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics. ICRA 2018 - International Conference on Robotics and Automation, May 2018, brisbane, Australia. ⟨hal-01768285⟩

Share

Metrics

Record views

164

Files downloads

178