Sparse single-index model

Abstract : Let $(\bX, Y)$ be a random pair taking values in $\mathbb R^p \times \mathbb R$. In the so-called single-index model, one has $Y=f^{\star}(\theta^{\star T}\bX)+\bW$, where $f^{\star}$ is an unknown univariate measurable function, $\theta^{\star}$ is an unknown vector in $\mathbb R^d$, and $W$ denotes a random noise satisfying $\mathbb E[\bW|\bX]=0$. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes larger than the number of observations (''$p$ larger than $n$'' paradigm). To circumvent this difficulty, we consider the single-index model estimation problem from a sparsity perspective using a PAC-Bayesian approach. On the theoretical side, we offer a sharp oracle inequality, which is more powerful than the best known oracle inequalities for other common procedures of single-index recovery. The proposed method is implemented by means of the reversible jump Markov chain Monte Carlo technique and its performance is compared with that of standard procedures.
Type de document :
Article dans une revue
Journal of Machine Learning Research, Journal of Machine Learning Research, 2013, 14, pp.243−280
Liste complète des métadonnées


https://hal.inria.fr/hal-00556652
Contributeur : Pierre Alquier <>
Soumis le : mercredi 5 octobre 2011 - 11:47:17
Dernière modification le : lundi 29 mai 2017 - 14:26:25
Document(s) archivé(s) le : mardi 13 novembre 2012 - 15:12:00

Fichiers

singleindex6.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00556652, version 2
  • ARXIV : 1101.3229

Collections

Citation

Pierre Alquier, Gérard Biau. Sparse single-index model. Journal of Machine Learning Research, Journal of Machine Learning Research, 2013, 14, pp.243−280. <hal-00556652v2>

Partager

Métriques

Consultations de
la notice

540

Téléchargements du document

176