Boosted Fitted Q-Iteration

Samuele Tosatto 1 Matteo Pirotta 2 Carlo D'Eramo 1 Marcello Restelli 1
2 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : This paper is about the study of B-FQI, an Approximated Value Iteration (AVI) algorithm that exploits a boosting procedure to estimate the action-value function in reinforcement learning problems. B-FQI is an iterative off-line algorithm that, given a dataset of transitions, builds an approximation of the optimal action-value function by summing the approximations of the Bell-man residuals across all iterations. The advantage of such approach w.r.t. to other AVI methods is twofold: (1) while keeping the same function space at each iteration, B-FQI can represent more complex functions by considering an additive model; (2) since the Bellman residual decreases as the optimal value function is approached , regression problems become easier as iterations proceed. We study B-FQI both theoretically , providing also a finite-sample error upper bound for it, and empirically, by comparing its performance to the one of FQI in different domains and using different regression techniques.
Type de document :
Communication dans un congrès
34th International Conference on Machine Learning (ICML), Aug 2017, Sydney, Australia
Liste complète des métadonnées

Littérature citée [33 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01653332
Contributeur : Alessandro Lazaric <>
Soumis le : vendredi 1 décembre 2017 - 12:20:47
Dernière modification le : jeudi 11 janvier 2018 - 06:27:32

Fichier

tosatto17a.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01653332, version 1

Collections

Citation

Samuele Tosatto, Matteo Pirotta, Carlo D'Eramo, Marcello Restelli. Boosted Fitted Q-Iteration. 34th International Conference on Machine Learning (ICML), Aug 2017, Sydney, Australia. 〈hal-01653332〉

Partager

Métriques

Consultations de la notice

55

Téléchargements de fichiers

11