Skip to Main content Skip to Navigation

A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors

Anthony Lee 1 Francois Caron 2, 3 Arnaud Doucet 4 Chris Holmes 1 
2 ALEA - Advanced Learning Evolutionary Algorithms
Inria Bordeaux - Sud-Ouest, UB - Université de Bordeaux, CNRS - Centre National de la Recherche Scientifique : UMR5251
Abstract : Variable selection techniques have become increasingly popular amongst statisticians due to an increased number of regression and classification applications involving high-dimensional data where we expect some predictors to be unimportant. In this context, Bayesian variable selection techniques involving Markov chain Monte Carlo exploration of the posterior distribution over models can be prohibitively computationally expensive and so there has been attention paid to quasi-Bayesian approaches such as maximum a posteriori (MAP) estimation using priors that induce sparsity in such estimates. We focus on this latter approach, expanding on the hierarchies proposed to date to provide a Bayesian interpretation and generalization of state-of-the-art penalized optimization approaches and providing simultaneously a natural way to include prior information about parameters within this framework. We give examples of how to use this hierarchy to compute MAP estimates for linear and logistic regression as well as sparse precision-matrix estimates in Gaussian graphical models. In addition, an adaptive group lasso method is derived using the framework.
Complete list of metadata
Contributor : Francois Caron Connect in order to contact the contributor
Submitted on : Friday, September 17, 2010 - 12:05:57 PM
Last modification on : Friday, February 4, 2022 - 3:23:33 AM

Links full text


  • HAL Id : inria-00518430, version 1
  • ARXIV : 1009.1914



Anthony Lee, Francois Caron, Arnaud Doucet, Chris Holmes. A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors. [Research Report] arXiv:1009.1914, 2010. ⟨inria-00518430⟩



Record views