hal-00243116, version 3

Data-driven calibration of penalties for least squares regression

Sylvain Arlot () ^1, 2, Pascal Massart () ^1, 2

• 1 :  Laboratoire de Mathématiques d'Orsay (LM-Orsay)
• http://www.math.u-psud.fr
  CNRS : UMR8628 – Université Paris XI - Paris Sud France
• 2 :  SELECT (INRIA Futurs)
• 
  INRIA – Université Paris XI - Paris Sud France

Références bibliographiques

• Type de publication : Documents sans référence de publication (Preprint)
• Domaine :

  Mathématiques/Statistiques

  Statistiques/Théorie

  Statistiques/Machine Learning

  Statistiques/Méthodologie

• Titre : Data-driven calibration of penalties for least squares regression
• Résumé : Penalization procedures often suffer from their dependence on multiplying factors, whose optimal values are either unknown or hard to estimate from the data. We propose a completely data-driven calibration algorithm for this parameter in the least squares regression framework, without assuming a particular shape for the penalty. Our algorithm relies on the concept of minimal penalty, recently introduced by Birge and Massart (2007) in the context of penalized least squares for Gaussian homoscedastic regression. On the positive side, the minimal penalty can be evaluated from the data themselves, leading to a data-driven estimation of an optimal penalty which can be used in practice; on the negative side, their approach heavily relies on the homoscedastic Gaussian nature of their stochastic framework. The purpose of this paper is twofold: stating a more general heuristics for designing a data-driven penalty (the slope heuristics) and proving that it works for penalized least squares random design regression, even for heteroscedastic non-Gaussian data. For some technical reasons, some exact mathematical results will be proved only for regressogram bin-width selection. This is at least a first step towards further results, since the approach and the method that we use are indeed general.
• Langue du texte
  intégral : Anglais
• Mots Clés : Data-driven calibration – Non-parametric regression – Model selection by penalization – Heteroscedastic data – Regressogram
• Classification : AMS primary 62G05 ; secondary 62J05

Liste des fichiers attachés à ce document :

TEX

AM_pente.tex(113.4 KB)

jmlr2e.sty(12.7 KB)

pente.bib(17.9 KB)

publi_arlot_en.bib(6.4 KB)

S1_pente_176_2meth.eps(9.5 KB)

S1_pente_176_2meth.jpg(148.9 KB)

S1_pente_510_2meth.eps(9.5 KB)

S1_pente_510_2meth.jpg(151.1 KB)

AM_pente.bbl(10.8 KB)

PS

AM_pente.ps(349 KB)

PDF

AM_pente.pdf(486.6 KB)

• hal-00243116, version 3
• http://hal.archives-ouvertes.fr/hal-00243116
• oai:hal.archives-ouvertes.fr:hal-00243116
• Contributeur : Sylvain Arlot <>
• Soumis le : Vendredi 19 Septembre 2008, 10:31:52
• Dernière modification le : Vendredi 19 Septembre 2008, 10:38:57