Rate of Convergence and Error Bounds for LSTD($\lambda$) - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Rapport (Rapport De Recherche) Année : 2014

Rate of Convergence and Error Bounds for LSTD($\lambda$)

Manel Tagorti
  • Fonction : Auteur
  • PersonId : 948730
Bruno Scherrer

Résumé

We consider LSTD($\lambda$), the least-squares temporal-difference algorithm with eligibility traces algorithm proposed by Boyan (2002). It computes a linear approximation of the value function of a fixed policy in a large Markov Decision Process. Under a $\beta$-mixing assumption, we derive, for any value of $\lambda \in (0,1)$, a high-probability estimate of the rate of convergence of this algorithm to its limit. We deduce a high-probability bound on the error of this algorithm, that extends (and slightly improves) that derived by Lazaric et al. (2012) in the specific case where $\lambda=0$. In particular, our analysis sheds some light on the choice of $\lambda$ with respect to the quality of the chosen linear space and the number of samples, that complies with simulations.
Fichier principal
Vignette du fichier
report.pdf (455.47 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00990525 , version 1 (13-05-2014)

Identifiants

Citer

Manel Tagorti, Bruno Scherrer. Rate of Convergence and Error Bounds for LSTD($\lambda$). [Research Report] 2014. ⟨hal-00990525⟩
275 Consultations
395 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More