Skip to Main content Skip to Navigation
Conference papers

Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Valentina Zantedeschi 1, 2 Paul Viallard 3 Emilie Morvant 3 Rémi Emonet 3 Amaury Habrard 3 Pascal Germain 4 Benjamin Guedj 5, 6, 1, 2 
1 MODAL - MOdel for Data Analysis and Learning
LPP - Laboratoire Paul Painlevé - UMR 8524, Université de Lille, Sciences et Technologies, Inria Lille - Nord Europe, METRICS - Evaluation des technologies de santé et des pratiques médicales - ULR 2694, Polytech Lille - École polytechnique universitaire de Lille
Abstract : We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives-both with uninformed (data-independent) and informed (data-dependent) priors.
Complete list of metadata

https://hal.inria.fr/hal-03278470
Contributor : Benjamin Guedj Connect in order to contact the contributor
Submitted on : Monday, July 5, 2021 - 3:54:16 PM
Last modification on : Saturday, June 25, 2022 - 9:24:27 AM

File

2106.12535.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03278470, version 1
  • ARXIV : 2106.12535

Citation

Valentina Zantedeschi, Paul Viallard, Emilie Morvant, Rémi Emonet, Amaury Habrard, et al.. Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound. NeurIPS, 2021, Online, France. ⟨hal-03278470⟩

Share

Metrics

Record views

40

Files downloads

41