Skip to Main content Skip to Navigation
Conference papers

High-Accuracy Value-Function Approximation with Neural Networks Applied to the Acrobot

Rémi Coulom 1
1 CORTEX - Neuromimetic intelligence
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Several reinforcement-learning techniques have already been applied to the Acrobot control problem, using linear function approximators to estimate the value function. In this paper, we present experimental results obtained by using a feedforward neural network instead. The learning algorithm used was model-based continuous TD(lambda). It generated an efficient controller, producing a high-accuracy state-value function. A striking feature of this value function is a very sharp 4-dimensional ridge that is extremely hard to evaluate with linear parametric approximators. From a broader point of view, this experimental success demonstrates some of the qualities of feedforward neural networks in comparison with linear approximators in reinforcement learning.
Document type :
Conference papers
Complete list of metadatas

Cited literature [10 references]  Display  Hide  Download

https://hal.inria.fr/inria-00107776
Contributor : Publications Loria <>
Submitted on : Thursday, October 19, 2006 - 9:08:54 AM
Last modification on : Thursday, January 11, 2018 - 6:19:48 AM
Long-term archiving on: : Wednesday, March 29, 2017 - 1:04:28 PM

Identifiers

  • HAL Id : inria-00107776, version 1

Collections

Citation

Rémi Coulom. High-Accuracy Value-Function Approximation with Neural Networks Applied to the Acrobot. 12th European Symposium on Artificial Neural Networks - ESANN'2004, Michel Verleysen, 2004, Bruges, Belgique, pp.7-12. ⟨inria-00107776⟩

Share

Metrics

Record views

204

Files downloads

380