Skip to Main content Skip to Navigation
New interface
Conference papers

High-Accuracy Value-Function Approximation with Neural Networks Applied to the Acrobot

Rémi Coulom 1 
1 CORTEX - Neuromimetic intelligence
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Several reinforcement-learning techniques have already been applied to the Acrobot control problem, using linear function approximators to estimate the value function. In this paper, we present experimental results obtained by using a feedforward neural network instead. The learning algorithm used was model-based continuous TD(lambda). It generated an efficient controller, producing a high-accuracy state-value function. A striking feature of this value function is a very sharp 4-dimensional ridge that is extremely hard to evaluate with linear parametric approximators. From a broader point of view, this experimental success demonstrates some of the qualities of feedforward neural networks in comparison with linear approximators in reinforcement learning.
Document type :
Conference papers
Complete list of metadata

Cited literature [10 references]  Display  Hide  Download
Contributor : Publications Loria Connect in order to contact the contributor
Submitted on : Thursday, October 19, 2006 - 9:08:54 AM
Last modification on : Friday, February 4, 2022 - 3:31:27 AM
Long-term archiving on: : Wednesday, March 29, 2017 - 1:04:28 PM


  • HAL Id : inria-00107776, version 1



Rémi Coulom. High-Accuracy Value-Function Approximation with Neural Networks Applied to the Acrobot. 12th European Symposium on Artificial Neural Networks - ESANN'2004, Michel Verleysen, 2004, Bruges, Belgique, pp.7-12. ⟨inria-00107776⟩



Record views


Files downloads