Sequential Transfer in Multi-armed Bandit with Finite Set of Models

Mohammad Gheshlaghi Azar 1 Alessandro Lazaric 2, 3 Emma Brunskill 1
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may significantly improve the learning performance, most of the literature on transfer is focused on batch learning tasks. In this paper we study the problem of \textit{sequential transfer in online learning}, notably in the multi--armed bandit framework, where the objective is to minimize the total regret over a sequence of tasks by transferring knowledge from prior tasks. Under the assumption that the tasks are drawn from a stationary distribution over a finite set of models, we define a novel bandit algorithm based on a method-of-moments approach for the estimation of the possible tasks and derive regret bounds for it. We introduce a novel bandit algorithm based on a method-of-moments approach for estimating the possible tasks and derive regret bounds for it. Finally, we report preliminary empirical results confirming the theoretical findings.
Type de document :
Communication dans un congrès
NIPS - Advances in Neural Information Processing Systems 25 - 2013, Dec 2013, Lake Tahoe, United States. 2013
Liste complète des métadonnées

Littérature citée [21 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00924025
Contributeur : Alessandro Lazaric <>
Soumis le : lundi 6 janvier 2014 - 11:05:19
Dernière modification le : jeudi 11 janvier 2018 - 06:22:13
Document(s) archivé(s) le : jeudi 10 avril 2014 - 16:25:39

Fichier

transfer-bandit.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00924025, version 1

Citation

Mohammad Gheshlaghi Azar, Alessandro Lazaric, Emma Brunskill. Sequential Transfer in Multi-armed Bandit with Finite Set of Models. NIPS - Advances in Neural Information Processing Systems 25 - 2013, Dec 2013, Lake Tahoe, United States. 2013. 〈hal-00924025〉

Partager

Métriques

Consultations de la notice

302

Téléchargements de fichiers

268