Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

PAC-Bayes unleashed: generalisation bounds with unbounded losses

Maxime Haddouche 1, 2 Benjamin Guedj 3, 4, 5, 6, 2 Omar Rivasplata 4, 3, 2 John Shawe-Taylor 4, 3, 2
6 MODAL - MOdel for Data Analysis and Learning
Inria Lille - Nord Europe, LPP - Laboratoire Paul Painlevé - UMR 8524, METRICS - Evaluation des technologies de santé et des pratiques médicales - ULR 2694, Polytech Lille - École polytechnique universitaire de Lille, Université de Lille, Sciences et Technologies
Abstract : We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems where the loss function is bounded (typically assumed to take values in the interval [0;1]). In order to relax this assumption, we propose a new notion called the \emph{special boundedness condition}, which effectively allows the range of the loss to depend on each predictor. Based on this new notion we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.
Complete list of metadata

Cited literature [26 references]  Display  Hide  Download

https://hal.inria.fr/hal-02872173
Contributor : Benjamin Guedj <>
Submitted on : Wednesday, June 17, 2020 - 4:00:53 PM
Last modification on : Tuesday, February 23, 2021 - 3:28:06 AM

File

2006.07279.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02872173, version 1
  • ARXIV : 2006.07279

Citation

Maxime Haddouche, Benjamin Guedj, Omar Rivasplata, John Shawe-Taylor. PAC-Bayes unleashed: generalisation bounds with unbounded losses. 2020. ⟨hal-02872173⟩

Share

Metrics

Record views

43

Files downloads

459