Online Learning with Noise: A Kernel-Based Policy-Gradient Approach

Emmanuel Daucé 1 Alain Dutech 2
2 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Various forms of noise are present in the brain. The role of noise in a exploration/exploitation trade-off is cast into the framework of reinforcement learning for a complex task of motor learning. A neuro-controler using a linear transformation of the input to which is added a gaussian noise is modelized as a stochastic controler that can be learned online in ''direct policy-gradient'' scheme. The reward signal is related to sensor information, thus no direct or indirect model of the system to control is needed. The task chosen (reaching with a multi-joint arm) is redundant and non-linear. The controler inputs are then projected to a feature space of higher dimension using a topographic coding based on gaussian kernels. We show that through a consistent noise level it possible to explore the environnment so as to find good control solution that can be exploited. Besides, the controler is able to adapt continuously to changes in the system dynamics. The general framework of this work will allow to study various noises and their effect, especially since it is quite compatible with more complexe types of stochastic neuro-controler, as demonstrated by other works on binary or spiking networks.
Type de document :
Communication dans un congrès
Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. 2010
Liste complète des métadonnées

Littérature citée [23 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00517006
Contributeur : Alain Dutech <>
Soumis le : lundi 13 septembre 2010 - 13:31:07
Dernière modification le : jeudi 18 janvier 2018 - 01:52:03
Document(s) archivé(s) le : mardi 23 octobre 2012 - 16:00:36

Fichier

dauce_noisePolicyGradient_Neur...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00517006, version 1

Citation

Emmanuel Daucé, Alain Dutech. Online Learning with Noise: A Kernel-Based Policy-Gradient Approach. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. 2010. 〈inria-00517006〉

Partager

Métriques

Consultations de la notice

279

Téléchargements de fichiers

103