Sparse Temporal Difference Learning using LASSO

Manuel Loth 1 Manuel Davy 1, 2 Philippe Preux 1
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
2 LAGIS-SI
LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We consider the problem of on-line value function estimation in reinforcement learning. We concentrate on the function approximator to use. To try to break the curse of dimensionality, we focus on non parametric function approximators. We propose to fit the use of kernels into the temporal difference algorithms by using regression via the LASSO. We introduce the equi-gradient descent algorithm (EGD) which is a direct adaptation of the one recently introduced in the LARS algorithm family for solving the LASSO. We advocate our choice of the EGD as a judicious algorithm for these tasks. We present the EGD algorithm in details as well as some experimental results. We insist on the qualities of the EGD for reinforcement learning.
Type de document :
Communication dans un congrès
IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, Apr 2007, Hawaï, USA, United States. 2007
Liste complète des métadonnées

Littérature citée [17 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00117075
Contributeur : Manuel Loth <>
Soumis le : jeudi 30 novembre 2006 - 13:15:51
Dernière modification le : jeudi 11 janvier 2018 - 06:26:40
Document(s) archivé(s) le : mardi 6 avril 2010 - 23:39:31

Fichier

lassoTd.pdf
Fichiers éditeurs autorisés sur une archive ouverte

Identifiants

  • HAL Id : inria-00117075, version 1

Collections

Citation

Manuel Loth, Manuel Davy, Philippe Preux. Sparse Temporal Difference Learning using LASSO. IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, Apr 2007, Hawaï, USA, United States. 2007. 〈inria-00117075〉

Partager

Métriques

Consultations de la notice

349

Téléchargements de fichiers

484