Skip to Main content Skip to Navigation
Journal articles

The recursive variational Gaussian approximation (R-VGA)

Marc Lambert 1, 2 Silvere Bonnabel 3, 4 Francis Bach 2 
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique - ENS Paris, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We consider the problem of computing a Gaussian approximation to the posterior distribution of a parameter given N observations and a Gaussian prior. Owing to the need of processing large sample sizes N, a variety of approximate tractable methods revolving around online learning have flourished over the past decades. In the present work, we propose to use variational inference (VI) to compute a Gaussian approximation to the posterior through a single pass over the data. Our algorithm is a recursive version of the variational Gaussian approximation we have called recursive variational Gaussian approximation (R-VGA). We start from the prior, and for each observation we compute the nearest Gaussian approximation in the sense of Kullback-Leibler divergence to the posterior given this observation. In turn, this approximation is considered as the new prior when incorporating the next observation. This recursive version based on a sequence of optimal Gaussian approximations leads to a novel implicit update scheme which resembles the online Newton algorithm, and which is shown to boil down to the Kalman filter for Bayesian linear regression. In the context of Bayesian logistic regression the implicit scheme may be solved, and the algorithm is shown to perform better than the extended Kalman filter, while being far less computationally demanding than its sampling counterparts.
Complete list of metadata

https://hal.inria.fr/hal-03086627
Contributor : Marc Lambert Connect in order to contact the contributor
Submitted on : Tuesday, December 7, 2021 - 9:13:34 PM
Last modification on : Wednesday, June 15, 2022 - 4:25:31 AM

File

RVGA-HAL-v2.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03086627, version 2

Citation

Marc Lambert, Silvere Bonnabel, Francis Bach. The recursive variational Gaussian approximation (R-VGA). Statistics and Computing, Springer Verlag (Germany), In press. ⟨hal-03086627v2⟩

Share

Metrics

Record views

3868

Files downloads

633