HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Journal articles

Asynchronous Neurocomputing for optimal control and reinforcement learning with large state spaces

Bruno Scherrer 1
1 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : We consider two machine learning related problems, optimal control and reinforcement learning. We show that, even when their state space is very large (possibly infinite), natural algorithmic solutions can be implemented in an asynchronous neurocomputing way, that is by an assembly of interconnected simple neuron-like units which does not require any synchronization. From a neuroscience perspective, this work might help understanding how an asynchronous assembly of simple units can give rise to efficient control. From a computational point of view, such neurocomputing architectures can exploit their massively parallel structure and be significantly faster than standard sequential approaches. The contributions of this paper are the following: 1) We introduce a theoretically sound methodology for designing a whole class of asynchronous neurocomputing algorithms. 2) We build an original asynchronous neurocomputing architecture for optimal control in a small state space, then we show how to improve this architecture so that also solves the reinforcement learning problem. 3) Finally, we show how to extend this architecture to address the case where the state space is large (possibly infinite) by using an asynchronous neurocomputing adaptive approximation scheme. We illustrate this approximation scheme on two continuous space control problems.
Complete list of metadata

Contributor : Bruno Scherrer Connect in order to contact the contributor
Submitted on : Tuesday, November 15, 2005 - 1:06:36 PM
Last modification on : Friday, February 26, 2021 - 3:28:04 PM
Long-term archiving on: : Friday, April 2, 2010 - 7:19:56 PM


  • HAL Id : inria-00000722, version 1



Bruno Scherrer. Asynchronous Neurocomputing for optimal control and reinforcement learning with large state spaces. Neurocomputing, Elsevier, 2005, New Aspects in Neurocomputing: 11th European Symposium on Artificial Neural Networks., 23, pp.229-251. ⟨inria-00000722⟩



Record views


Files downloads