Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

The recursive variational Gaussian approximation (R-VGA)

Marc Lambert 1, 2 Silvere Bonnabel 3, 4 Francis Bach 2
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We consider the problem of computing a Gaussian approximation to the posterior distribution of a parameter given N observations and a Gaussian prior. Owing to the need of processing large sample sizes N, a variety of approximate tractable methods revolving around online learning have flourished over the past decades. In the present work, we propose to use variational inference (VI) to compute a Gaussian approximation to the posterior through a single pass over the data. Our algorithm is a recursive version of the variational Gaussian approximation we have called recursive variational Gaussian approximation (R-VGA). We start from the prior, and for each observation we compute the nearest Gaussian approximation in the sense of Kullback-Leibler divergence to the posterior given this observation. In turn, this approximation is considered as the new prior when incorporating the next observation. This recursive version based on a sequence of optimal Gaussian approximations leads to a novel implicit update scheme which resembles the online Newton algorithm, and which is shown to boil down to the Kalman filter for Bayesian linear regression. In the context of Bayesian logistic regression the implicit scheme may be solved, and the algorithm is shown to perform better than the extended Kalman filter, while being far less computationally demanding than its sampling counterparts.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas
Contributor : Marc Lambert <>
Submitted on : Wednesday, December 30, 2020 - 3:08:35 PM
Last modification on : Thursday, January 14, 2021 - 3:14:12 PM


Files produced by the author(s)


  • HAL Id : hal-03086627, version 1


Marc Lambert, Silvere Bonnabel, Francis Bach. The recursive variational Gaussian approximation (R-VGA). 2020. ⟨hal-03086627⟩



Record views


Files downloads