PAC-Bayesian Bounds based on the Rényi Divergence

Luc Bégin 1 Pascal Germain 2 François Laviolette 3 Jean-Francis Roy 3
1 Université de Moncton
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
3 Department of Computer Science and Software Engineering [Québec]
Abstract : We propose a simplified proof process for PAC-Bayesian generalization bounds, that allows to divide the proof in four successive inequalities, easing the "customization" of PAC-Bayesian theorems. We also propose a family of PAC-Bayesian bounds based on the Rényi divergence between the prior and posterior distributions, whereas most PAC-Bayesian bounds are based on the Kullback-Leibler divergence. Finally, we present an empirical evaluation of the tightness of each inequality of the simplified proof, for both the classical PAC-Bayesian bounds and those based on the Rényi divergence.
Keywords : Statistical Learning Theory PAC-Bayes
Type de document :
Communication dans un congrès
International Conference on Artificial Intelligence and Statistics (AISTATS 2016), May 2016, Cadiz, Spain. 2016, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. 〈http://www.aistats.org/aistats2016/〉
Liste complète des métadonnées

Littérature citée [31 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-01384783
Contributeur : Pascal Germain <>
Soumis le : jeudi 20 octobre 2016 - 15:07:50
Dernière modification le : jeudi 26 avril 2018 - 10:28:58

Fichier

begin16.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01384783, version 1

Collections

ENS-PARIS | INRIA | PSL | INRIA-ROCQ

Citation

Luc Bégin, Pascal Germain, François Laviolette, Jean-Francis Roy. PAC-Bayesian Bounds based on the Rényi Divergence. International Conference on Artificial Intelligence and Statistics (AISTATS 2016), May 2016, Cadiz, Spain. 2016, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. 〈http://www.aistats.org/aistats2016/〉. 〈hal-01384783〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

167

Téléchargements de fichiers

125

Contact


archive-ouverte@inria.fr