LSPI with Random Projections

Mohammad Ghavamzadeh 1 Alessandro Lazaric 1 Odalric Maillard 1 Rémi Munos 1
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We consider the problem of reinforcement learning in high-dimensional spaces when the number of features is bigger than the number of samples. In particular, we study the least-squares temporal difference (LSTD) learning algorithm when a space of low dimension is generated with a random projection from a high-dimensional space. We provide a thorough theoretical analysis of the LSTD with random projections and derive performance bounds for the resulting algorithm. We also show how the error of LSTD with random projections is propagated through the iterations of a policy iteration algorithm and provide a performance bound for the resulting least-squares policy iteration (LSPI) algorithm.
Document type :
Reports
Complete list of metadatas

Cited literature [21 references]  Display  Hide  Download

https://hal.inria.fr/inria-00530762
Contributor : Mohammad Ghavamzadeh <>
Submitted on : Friday, October 29, 2010 - 6:23:43 PM
Last modification on : Thursday, February 21, 2019 - 10:52:49 AM
Long-term archiving on : Friday, October 26, 2012 - 12:45:29 PM

File

randproj-lspi.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00530762, version 1

Collections

Citation

Mohammad Ghavamzadeh, Alessandro Lazaric, Odalric Maillard, Rémi Munos. LSPI with Random Projections. [Technical Report] 2010. ⟨inria-00530762⟩

Share

Metrics

Record views

468

Files downloads

216