Adaptive Bandits: Towards the best history-dependent strategy

Odalric-Ambrym Maillard 1 Rémi Munos 1
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We consider multi-armed bandit games with possibly adaptive opponents. We introduce models Theta of constraints based on equivalence classes on the common history (information shared by the player and the opponent) which dene two learning scenarios: (1) The opponent is constrained, i.e. he provides rewards that are stochastic functions of equivalence classes dened by some model theta*\in Theta. The regret is measured with respect to (w.r.t.) the best history-dependent strategy. (2) The opponent is arbitrary and we measure the regret w.r.t. the best strategy among all mappings from classes to actions (i.e. the best history-class-based strategy) for the best model in Theta. This allows to model opponents (case 1) or strategies (case 2) which handles nite memory, periodicity, standard stochastic bandits and other situations. When Theta={theta}, i.e. only one model is considered, we derive tractable algorithms achieving a tight regret (at time T) bounded by ~O(sqrt(TAC)), where C is the number of classes of theta. Now, when many models are available, all known algorithms achieving a nice regret O(sqrt(T)) are unfortunately not tractable and scale poorly with the number of models |Theta|. Our contribution here is to provide tractable algorithms with regret bounded by T^{2/3}C^{1/3} log(|Theta|)^{1/2}.
Type de document :
Rapport
[Technical Report] 2011, pp.14
Liste complète des métadonnées

Littérature citée [18 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00574999
Contributeur : Odalric-Ambrym Maillard <>
Soumis le : mercredi 9 mars 2011 - 13:57:08
Dernière modification le : jeudi 11 janvier 2018 - 06:22:13
Document(s) archivé(s) le : mardi 6 novembre 2012 - 15:56:07

Fichier

AdaptiveBandits_HAL.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00574999, version 1

Collections

Citation

Odalric-Ambrym Maillard, Rémi Munos. Adaptive Bandits: Towards the best history-dependent strategy. [Technical Report] 2011, pp.14. 〈inria-00574999〉

Partager

Métriques

Consultations de la notice

580

Téléchargements de fichiers

223