Regret Bounds for Reinforcement Learning with Policy Advice

Mohammad Gheshlaghi Azar; Alessandro Lazaric; Emma Brunskill

Regret Bounds for Reinforcement Learning with Policy Advice

Mohammad Gheshlaghi Azar ¹ Alessandro Lazaric ^{2, 3} Emma Brunskill ¹

1 Computer Science Department - Carnegie Mellon University

LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal

3 LIFL - Laboratoire d'Informatique Fondamentale de Lille

Abstract : In some reinforcement learning problems an agent may be provided with a set of input policies, perhaps learned from prior experience or provided by advisors. We present a reinforcement learning with policy advice (RLPA) algorithm which leverages this input set and learns to use the best policy in the set for the reinforcement learning task at hand. We prove that RLPA has a sub-linear regret of $\widetilde O(\sqrt{T})$ relative to the best input policy, and that both this regret and its computational complexity are independent of the size of the state and action space. Our empirical simulations support our theoretical analysis. This suggests RLPA may offer significant advantages in large domains where some prior good policies are provided.

Document type :

Conference papers

Domain :

Computer Science [cs] / Machine Learning [cs.LG]

Complete list of metadatas

Cited literature [15 references] Display Hide Download

https://hal.inria.fr/hal-00924021
Contributor : Alessandro Lazaric <>
Submitted on : Monday, January 6, 2014 - 11:00:27 AM
Last modification on : Thursday, August 22, 2019 - 12:10:38 PM
Long-term archiving on : Thursday, April 10, 2014 - 4:25:17 PM

RLPAcr.pdf

Files produced by the author(s)

Regret Bounds for Reinforcement Learning with Policy Advice

File

Identifiers

Collections

Citation

Export

Metrics

642

515

Contact

Regret Bounds for Reinforcement Learning with Policy Advice

File

Identifiers

Collections

Citation

Export

Share

Metrics

642

515

Contact