Skip to Main content Skip to Navigation
Conference papers

Model-Free Reinforcement Learning with Continuous Action in Practice

Thomas Degris 1 Patrick M. Pilarski 2 Richard S. Sutton 2 
Department of Computing Science [Edmonton]
Abstract : Reinforcement learning methods are often con- sidered as a potential solution to enable a robot to adapt to changes in real time to an unpredictable environment. However, with continuous action, only a few existing algorithms are practical for real-time learning. In such a setting, most effective methods have used a parameterized policy structure, often with a separate parameterized value function. The goal of this paper is to assess such actor-critic methods to form a fully specified practical algorithm. Our specific contributions include 1) developing the extension of existing incremental policy-gradient algorithms to use eligibility traces, 2) an empir- ical comparison of the resulting algorithms using continuous actions, 3) the evaluation of a gradient-scaling technique that can significantly improve performance. Finally, we apply our actor-critic algorithm to learn on a robotic platform with a fast sensorimotor cycle (10ms). Overall, these results constitute an important step towards practical real-time learning control with continuous action.
Document type :
Conference papers
Complete list of metadata

Cited literature [17 references]  Display  Hide  Download
Contributor : Thomas Degris Connect in order to contact the contributor
Submitted on : Wednesday, December 12, 2012 - 4:32:05 PM
Last modification on : Saturday, March 26, 2022 - 3:18:10 AM
Long-term archiving on: : Sunday, December 18, 2016 - 12:21:21 AM


Files produced by the author(s)


  • HAL Id : hal-00764281, version 1



Thomas Degris, Patrick M. Pilarski, Richard S. Sutton. Model-Free Reinforcement Learning with Continuous Action in Practice. American Control Conference, Jun 2012, Montreal, Canada. ⟨hal-00764281⟩



Record views


Files downloads