Regret Bounds for Reinforcement Learning with Policy Advice

Mohammad Gheshlaghi Azar 1 Alessandro Lazaric 2, 3 Emma Brunskill 1
1 Computer Science Department - Carnegie Mellon University
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
3 LIFL - Laboratoire d'Informatique Fondamentale de Lille
Abstract : In some reinforcement learning problems an agent may be provided with a set of input policies, perhaps learned from prior experience or provided by advisors. We present a reinforcement learning with policy advice (RLPA) algorithm which leverages this input set and learns to use the best policy in the set for the reinforcement learning task at hand. We prove that RLPA has a sub-linear regret of $\widetilde O(\sqrt{T})$ relative to the best input policy, and that both this regret and its computational complexity are independent of the size of the state and action space. Our empirical simulations support our theoretical analysis. This suggests RLPA may offer significant advantages in large domains where some prior good policies are provided.
Type de document :
Communication dans un congrès
ECML/PKDD - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2013, Prague, Czech Republic. 2013
Liste complète des métadonnées

Littérature citée [15 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-00924021
Contributeur : Alessandro Lazaric <>
Soumis le : lundi 6 janvier 2014 - 11:00:27
Dernière modification le : jeudi 11 janvier 2018 - 06:22:13
Document(s) archivé(s) le : jeudi 10 avril 2014 - 16:25:17

Fichier

RLPAcr.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00924021, version 1

Collections

UNIV-LILLE3 | INRIA | LAGIS | CRISTAL | CRISTAL-SEQUEL

Citation

Mohammad Gheshlaghi Azar, Alessandro Lazaric, Emma Brunskill. Regret Bounds for Reinforcement Learning with Policy Advice. ECML/PKDD - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2013, Prague, Czech Republic. 2013. 〈hal-00924021〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

373

Téléchargements de fichiers

289

Contact


archive-ouverte@inria.fr