Skip to Main content Skip to Navigation
Conference papers

Online Learning with Noise: A Kernel-Based Policy-Gradient Approach

Emmanuel Daucé 1 Alain Dutech 2 
2 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Various forms of noise are present in the brain. The role of noise in a exploration/exploitation trade-off is cast into the framework of reinforcement learning for a complex task of motor learning. A neuro-controler using a linear transformation of the input to which is added a gaussian noise is modelized as a stochastic controler that can be learned online in ''direct policy-gradient'' scheme. The reward signal is related to sensor information, thus no direct or indirect model of the system to control is needed. The task chosen (reaching with a multi-joint arm) is redundant and non-linear. The controler inputs are then projected to a feature space of higher dimension using a topographic coding based on gaussian kernels. We show that through a consistent noise level it possible to explore the environnment so as to find good control solution that can be exploited. Besides, the controler is able to adapt continuously to changes in the system dynamics. The general framework of this work will allow to study various noises and their effect, especially since it is quite compatible with more complexe types of stochastic neuro-controler, as demonstrated by other works on binary or spiking networks.
Complete list of metadata

Cited literature [23 references]  Display  Hide  Download
Contributor : Alain Dutech Connect in order to contact the contributor
Submitted on : Monday, September 13, 2010 - 1:31:07 PM
Last modification on : Wednesday, February 2, 2022 - 3:56:29 PM
Long-term archiving on: : Tuesday, October 23, 2012 - 4:00:36 PM


Files produced by the author(s)


  • HAL Id : inria-00517006, version 1


Emmanuel Daucé, Alain Dutech. Online Learning with Noise: A Kernel-Based Policy-Gradient Approach. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. ⟨inria-00517006⟩



Record views


Files downloads