Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

The recursive variational Gaussian approximation (R-VGA)

Marc Lambert 1, 2 Silvere Bonnabel 3, 4 Francis Bach 2
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We consider the problem of computing a Gaussian approximation to the posterior distribution of a parameter given N observations and a Gaussian prior. Owing to the need of processing large sample sizes N, a variety of approximate tractable methods revolving around online learning have flourished over the past decades. In the present work, we propose to use variational inference (VI) to compute a Gaussian approximation to the posterior through a single pass over the data. Our algorithm is a recursive version of the variational Gaussian approximation we have called recursive variational Gaussian approximation (R-VGA). We start from the prior, and for each observation we compute the nearest Gaussian approximation in the sense of Kullback-Leibler divergence to the posterior given this observation. In turn, this approximation is considered as the new prior when incorporating the next observation. This recursive version based on a sequence of optimal Gaussian approximations leads to a novel implicit update scheme which resembles the online Newton algorithm, and which is shown to boil down to the Kalman filter for Bayesian linear regression. In the context of Bayesian logistic regression the implicit scheme may be solved, and the algorithm is shown to perform better than the extended Kalman filter, while being far less computationally demanding than its sampling counterparts.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

https://hal.inria.fr/hal-03086627
Contributor : Marc Lambert <>
Submitted on : Wednesday, December 30, 2020 - 3:08:35 PM
Last modification on : Thursday, January 14, 2021 - 3:14:12 PM

File

RVGA-HAL.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03086627, version 1

Citation

Marc Lambert, Silvere Bonnabel, Francis Bach. The recursive variational Gaussian approximation (R-VGA). 2020. ⟨hal-03086627⟩

Share

Metrics

Record views

1036

Files downloads

408