Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Inverse Kinematics On-line Learning: a Kernel-Based Policy-Gradient approach

Emmanuel Daucé 1 Alain Dutech 2
2 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : In machine learning, ``kernel methods'' give a consistent framework for applying the perceptron algorithm to non-linear problems. In reinforcement learning, an analog of the perceptron delta-rule can be derived from the "policy-gradient" approach proposed by Williams in 1992 in the framework of stochastic neural networks. Despite its generality and straighforward applicability to continuous command problems, quite few developments of the method had been proposed since. Here we present an account of the use of a kernel transformation of the perception space for the \emph{on-line} learning of a motor command, in the case of eye orientation and multi-joint arm control. We show first that such a setting allows the system to solve non-linear problems, like the log-like resolution of a foveated retina, or the transformation from a cartesian perception space to the ``angular'' command of the multi-joint arm. More interestingly, the on-line recurrent learning we propose is simple and fully operant in changing environments, and allows for constant improvements of the politics, on the basis of simple and measurables error terms.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [14 references]  Display  Hide  Download

https://hal.inria.fr/inria-00520960
Contributor : Alain Dutech <>
Submitted on : Friday, September 24, 2010 - 3:49:55 PM
Last modification on : Wednesday, December 9, 2020 - 3:12:58 AM
Long-term archiving on: : Thursday, October 25, 2012 - 11:30:57 AM

File

draft_nips10.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00520960, version 1

Collections

Citation

Emmanuel Daucé, Alain Dutech. Inverse Kinematics On-line Learning: a Kernel-Based Policy-Gradient approach. 2010. ⟨inria-00520960⟩

Share

Metrics

Record views

316

Files downloads

376