Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Analyzing the discrepancy principle for kernelized spectral filter learning algorithms

Alain Celisse 1 Martin Wahl 2
1 MODAL - MOdel for Data Analysis and Learning
Inria Lille - Nord Europe, LPP - Laboratoire Paul Painlevé - UMR 8524, METRICS - Evaluation des technologies de santé et des pratiques médicales - ULR 2694, Polytech Lille - École polytechnique universitaire de Lille, Université de Lille, Sciences et Technologies
Abstract : We investigate the construction of early stopping rules in the non-parametric regression problem where iterative learning algorithms are used and the optimal iteration number is unknown. More precisely, we study the discrepancy principle, as well as modifications based on smoothed residuals, for kernelized spectral filter learning algorithms including gradient descent. Our main theoretical bounds are oracle inequalities established for the empirical estimation error (fixed design), and for the prediction error (random design). From these finite-sample bounds it follows that the classical discrepancy principle is statistically adaptive for slow rates occurring in the hard learning scenario, while the smoothed discrepancy principles are adaptive over ranges of faster rates (resp. higher smoothness parameters). Our approach relies on deviation inequalities for the stopping rules in the fixed design setting, combined with change-of-norm arguments to deal with the random design setting.
Complete list of metadata

Cited literature [58 references]  Display  Hide  Download

https://hal.inria.fr/hal-02548917
Contributor : Alain Celisse <>
Submitted on : Tuesday, April 21, 2020 - 9:25:41 AM
Last modification on : Wednesday, January 20, 2021 - 3:39:36 PM

Files

draft.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02548917, version 1

Collections

Citation

Alain Celisse, Martin Wahl. Analyzing the discrepancy principle for kernelized spectral filter learning algorithms. 2020. ⟨hal-02548917⟩

Share

Metrics

Record views

47

Files downloads

356