PAC-Bayesian Bounds based on the Rényi Divergence

Luc Bégin 1 Pascal Germain 2 François Laviolette 3 Jean-Francis Roy 3
1 Université de Moncton
2 SIERRA - Statistical Machine Learning and Parsimony
Inria de Paris, CNRS - Centre National de la Recherche Scientifique, ENS Paris - École normale supérieure - Paris, DI-ENS - Département d'informatique de l'École normale supérieure
3 Department of Computer Science and Software Engineering [Québec]
Abstract : We propose a simplified proof process for PAC-Bayesian generalization bounds, that allows to divide the proof in four successive inequalities, easing the "customization" of PAC-Bayesian theorems. We also propose a family of PAC-Bayesian bounds based on the Rényi divergence between the prior and posterior distributions, whereas most PAC-Bayesian bounds are based on the Kullback-Leibler divergence. Finally, we present an empirical evaluation of the tightness of each inequality of the simplified proof, for both the classical PAC-Bayesian bounds and those based on the Rényi divergence.
Keywords : Statistical Learning Theory PAC-Bayes
Type de document :
Communication dans un congrès
International Conference on Artificial Intelligence and Statistics (AISTATS 2016), May 2016, Cadiz, Spain. 2016, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. 〈http://www.aistats.org/aistats2016/〉
Liste complète des métadonnées

Littérature citée [31 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-01384783
Contributeur : Pascal Germain <>
Soumis le : jeudi 20 octobre 2016 - 15:07:50
Dernière modification le : mardi 23 janvier 2018 - 10:04:50

Fichier

begin16.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01384783, version 1

Collections

INRIA | PSL

Citation

Luc Bégin, Pascal Germain, François Laviolette, Jean-Francis Roy. PAC-Bayesian Bounds based on the Rényi Divergence. International Conference on Artificial Intelligence and Statistics (AISTATS 2016), May 2016, Cadiz, Spain. 2016, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. 〈http://www.aistats.org/aistats2016/〉. 〈hal-01384783〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

144

Téléchargements de fichiers

114

Contact


archive-ouverte@inria.fr