Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling

Christophe Dupuy 1, 2 Francis Bach 3, 2
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We study parameter inference in large-scale latent variable models. We first propose an unified treatment of online inference for latent variable models from a non-canonical exponential family, and draw explicit links between several previously proposed frequentist or Bayesian methods. We then propose a novel inference method for the frequentist estimation of parameters, that adapts MCMC methods to online inference of latent variable models with the proper use of local Gibbs sampling. Then, for latent Dirich-let allocation,we provide an extensive set of experiments and comparisons with existing work, where our new approach outperforms all previously proposed methods. In particular, using Gibbs sampling for latent variable inference is superior to variational inference in terms of test log-likelihoods. Moreover, Bayesian inference through variational methods perform poorly, sometimes leading to worse fits with latent variables of higher dimensionality.
Type de document :
Pré-publication, Document de travail
Under submission in JMLR. 2016
Liste complète des métadonnées

https://hal.inria.fr/hal-01284900
Contributeur : Christophe Dupuy <>
Soumis le : mercredi 27 juillet 2016 - 11:23:33
Dernière modification le : mardi 24 avril 2018 - 17:20:15

Fichiers

DupuyBach16.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01284900, version 3
  • ARXIV : 1603.02644

Citation

Christophe Dupuy, Francis Bach. Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling. Under submission in JMLR. 2016. 〈hal-01284900v3〉

Partager

Métriques

Consultations de la notice

299

Téléchargements de fichiers

433