Skip to Main content Skip to Navigation
Journal articles

PAC-Bayes unleashed: generalisation bounds with unbounded losses

Maxime Haddouche 1, 2 Benjamin Guedj 3, 4, 5, 6, 2 Omar Rivasplata 4, 3, 2 John Shawe-Taylor 4, 3, 2 
6 MODAL - MOdel for Data Analysis and Learning
LPP - Laboratoire Paul Painlevé - UMR 8524, Université de Lille, Sciences et Technologies, Inria Lille - Nord Europe, METRICS - Evaluation des technologies de santé et des pratiques médicales - ULR 2694, Polytech Lille - École polytechnique universitaire de Lille
Abstract : We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems where the loss function is bounded (typically assumed to take values in the interval [0;1]). In order to relax this assumption, we propose a new notion called the \emph{special boundedness condition}, which effectively allows the range of the loss to depend on each predictor. Based on this new notion we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.
Complete list of metadata

Cited literature [26 references]  Display  Hide  Download

https://hal.inria.fr/hal-02872173
Contributor : Benjamin Guedj Connect in order to contact the contributor
Submitted on : Wednesday, June 17, 2020 - 4:00:53 PM
Last modification on : Friday, April 1, 2022 - 3:52:08 AM

File

2006.07279.pdf
Files produced by the author(s)

Identifiers

Citation

Maxime Haddouche, Benjamin Guedj, Omar Rivasplata, John Shawe-Taylor. PAC-Bayes unleashed: generalisation bounds with unbounded losses. Entropy, MDPI, 2021, ⟨10.3390/e23101330⟩. ⟨hal-02872173⟩

Share

Metrics

Record views

46

Files downloads

133