The recursive variational Gaussian approximation (R-VGA) - Archive ouverte HAL Access content directly
Journal Articles Statistics and Computing Year : 2021

The recursive variational Gaussian approximation (R-VGA)

(1, 2) , (3, 4) , (2)
1
2
3
4

Abstract

We consider the problem of computing a Gaussian approximation to the posterior distribution of a parameter given N observations and a Gaussian prior. Owing to the need of processing large sample sizes N, a variety of approximate tractable methods revolving around online learning have flourished over the past decades. In the present work, we propose to use variational inference (VI) to compute a Gaussian approximation to the posterior through a single pass over the data. Our algorithm is a recursive version of the variational Gaussian approximation we have called recursive variational Gaussian approximation (R-VGA). We start from the prior, and for each observation we compute the nearest Gaussian approximation in the sense of Kullback-Leibler divergence to the posterior given this observation. In turn, this approximation is considered as the new prior when incorporating the next observation. This recursive version based on a sequence of optimal Gaussian approximations leads to a novel implicit update scheme which resembles the online Newton algorithm, and which is shown to boil down to the Kalman filter for Bayesian linear regression. In the context of Bayesian logistic regression the implicit scheme may be solved, and the algorithm is shown to perform better than the extended Kalman filter, while being far less computationally demanding than its sampling counterparts.
Fichier principal
Vignette du fichier
RVGA-HAL-v2.pdf (1.32 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03086627 , version 1 (30-12-2020)
hal-03086627 , version 2 (07-12-2021)

Identifiers

Cite

Marc Lambert, Silvere Bonnabel, Francis Bach. The recursive variational Gaussian approximation (R-VGA). Statistics and Computing, In press, ⟨10.1007/s11222-021-10068-w⟩. ⟨hal-03086627v2⟩
3922 View
724 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More