Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics - Archive ouverte HAL Access content directly
Conference Papers Year :

Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics

(1) , (1)
1

Abstract

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms , which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recently introduced Black-DROPS algorithm exploits a black-box optimization algorithm to achieve both high data-efficiency and good computation times when several cores are used; nevertheless, like all model-based policy search approaches, Black-DROPS does not scale to high dimensional state/action spaces. In this paper, we introduce a new model learning procedure in Black-DROPS that leverages parameterized black-box priors to (1) scale up to high-dimensional systems, and (2) be robust to large inaccuracies of the prior information. We demonstrate the effectiveness of our approach with the " pendubot " swing-up task in simulation and with a physical hexapod robot (48D state space, 18D action space) that has to walk forward as fast as possible. The results show that our new algorithm is more data-efficient than previous model-based policy search algorithms (with and without priors) and that it can allow a physical 6-legged robot to learn new gaits in only 16 to 30 seconds of interaction time.
Fichier principal
Vignette du fichier
1709.06917.pdf (1.7 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01768285 , version 1 (17-04-2018)

Identifiers

Cite

Konstantinos Chatzilygeroudis, Jean-Baptiste Mouret. Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics. ICRA 2018 - International Conference on Robotics and Automation, May 2018, brisbane, Australia. ⟨hal-01768285⟩
129 View
187 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More