On Sparse, Spectral and Other Parameterizations of Binary Probabilistic Models

David Buchman 1, 2 Mark Schmidt 3, 4 Shakir Mohamed 1, 2 David Poole 1, 2 Nando De Freitas 1, 2
3 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : This paper studies issues relating to the parameterization of probability distributions over binary data sets. Several such parameterizations of models for binary data are known, including the Ising, generalized Ising, canonical and full parameterizations. We also discuss a parameterization that we call the "spectral parameterization", which has received significantly less coverage in existing literature. We provide this parameterization with a spectral interpretation by casting log- linear models in terms of orthogonal Walsh- Hadamard harmonic expansions. Using various standard and group sparse regularizers for structural learning, we provide a comprehensive theoretical and empirical comparison of these parameterizations. We show that the spectral parameterization, along with the canonical, has the best performance and sparsity levels, while the spectral does not depend on any particular reference state. The spectral interpretation also provides a new starting point for analyzing the statistics of binary data sets; we measure the magnitude of higher order interactions in the underlying distributions for several data sets.
Type de document :
Communication dans un congrès
AISTATS 2012 - 15th International Conference on Artificial Intelligence and Statistics, Apr 2012, La Palma, Spain. 2012, 〈http://jmlr.csail.mit.edu/proceedings/papers/v22/〉
Liste complète des métadonnées

https://hal.inria.fr/hal-00717714
Contributeur : Mark Schmidt <>
Soumis le : vendredi 13 juillet 2012 - 14:28:52
Dernière modification le : vendredi 25 mai 2018 - 12:02:06
Document(s) archivé(s) le : dimanche 14 octobre 2012 - 02:55:22

Fichier

2012_AISTATS_SpectralParameter...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00717714, version 1

Collections

Citation

David Buchman, Mark Schmidt, Shakir Mohamed, David Poole, Nando De Freitas. On Sparse, Spectral and Other Parameterizations of Binary Probabilistic Models. AISTATS 2012 - 15th International Conference on Artificial Intelligence and Statistics, Apr 2012, La Palma, Spain. 2012, 〈http://jmlr.csail.mit.edu/proceedings/papers/v22/〉. 〈hal-00717714〉

Partager

Métriques

Consultations de la notice

310

Téléchargements de fichiers

851