LSTD with Random Projections

M. Ghavamzadeh 1 A. Lazaric 1 O. A. Maillard 1 R. Munos 1
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We consider the problem of reinforcement learning in high-dimensional spaces when the number of features is bigger than the number of samples. In particular, we study the least-squares temporal difference (LSTD) learning algorithm when a space of low dimension is generated with a random projection from a high-dimensional space. We provide a thorough theoretical analysis of the LSTD with random projections and derive performance bounds for the resulting algorithm. We also show how the error of LSTD with random projections is propagated through the iterations of a policy iteration algorithm and provide a performance bound for the resulting least-squares policy iteration (LSPI) algorithm.
Type de document :
Communication dans un congrès
Advances in Neural Information Processing Systems, 2010, Granada, Spain. 2010
Liste complète des métadonnées
Contributeur : Philippe Preux <>
Soumis le : vendredi 7 février 2014 - 08:23:54
Dernière modification le : jeudi 11 janvier 2018 - 06:22:13


  • HAL Id : hal-00943120, version 1



M. Ghavamzadeh, A. Lazaric, O. A. Maillard, R. Munos. LSTD with Random Projections. Advances in Neural Information Processing Systems, 2010, Granada, Spain. 2010. 〈hal-00943120〉



Consultations de la notice