Asynchronous Neurocomputing for optimal control and reinforcement learning with large state spaces

Bruno Scherrer 1
1 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : We consider two machine learning related problems, optimal control and reinforcement learning. We show that, even when their state space is very large (possibly infinite), natural algorithmic solutions can be implemented in an asynchronous neurocomputing way, that is by an assembly of interconnected simple neuron-like units which does not require any synchronization. From a neuroscience perspective, this work might help understanding how an asynchronous assembly of simple units can give rise to efficient control. From a computational point of view, such neurocomputing architectures can exploit their massively parallel structure and be significantly faster than standard sequential approaches. The contributions of this paper are the following: 1) We introduce a theoretically sound methodology for designing a whole class of asynchronous neurocomputing algorithms. 2) We build an original asynchronous neurocomputing architecture for optimal control in a small state space, then we show how to improve this architecture so that also solves the reinforcement learning problem. 3) Finally, we show how to extend this architecture to address the case where the state space is large (possibly infinite) by using an asynchronous neurocomputing adaptive approximation scheme. We illustrate this approximation scheme on two continuous space control problems.
Complete list of metadatas
Contributor : Bruno Scherrer <>
Submitted on : Tuesday, November 15, 2005 - 1:06:36 PM
Last modification on : Thursday, January 11, 2018 - 6:19:50 AM
Long-term archiving on : Friday, April 2, 2010 - 7:19:56 PM


  • HAL Id : inria-00000722, version 1



Bruno Scherrer. Asynchronous Neurocomputing for optimal control and reinforcement learning with large state spaces. Neurocomputing, Elsevier, 2005, New Aspects in Neurocomputing: 11th European Symposium on Artificial Neural Networks., 23, pp.229-251. ⟨inria-00000722⟩



Record views


Files downloads