Faster ICA by preconditioning with Hessian approximations

Abstract : Independent Component Analysis (ICA) is a powerful technique for unsupervised data exploration that is widely used across fields such as neuroscience, astronomy, chemistry or biology. Linear ICA is a linear latent factor model, similar to sparse dictionary learning, that aims at discovering statistically independent sources from multivariate observations. ICA is a probabilistic generative model for which inference is classically done by maximum likelihood estimation. Estimating sources by maximum likelihood leads to a smooth non-convex optimization problem where the unknown is a matrix called the separating or unmixing matrix. As the gradient of the likelihood is available in closed form, first order gradient methods, stochastic or non-stochastic, are often employed despite a slow convergence such as in the Infomax algorithm. While the Hessian is known analytically, the cost of its computation and inversion makes Newton method unpractical for a large number of sources. We show how sparse and positive approximations of the true Hessian can be obtained and used to precondition the L-BFGS algorithm. Results on simulations and two applied problems (EEG data and image patches) demonstrate that the proposed technique leads to convergence that can be orders of magnitude faster than algorithms commonly used today even when looking for hundred of sources.
Type de document :
Pré-publication, Document de travail
2017
Liste complète des métadonnées


https://hal.inria.fr/hal-01552340
Contributeur : Pierre Ablin <>
Soumis le : lundi 3 juillet 2017 - 10:48:30
Dernière modification le : jeudi 6 juillet 2017 - 01:10:11

Fichier

quasi-newton-methods (6).pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01552340, version 1
  • ARXIV : 1706.08171

Collections

Citation

Pierre Ablin, Jean-Francois Cardoso, Alexandre Gramfort. Faster ICA by preconditioning with Hessian approximations. 2017. <hal-01552340>

Partager

Métriques

Consultations de
la notice

66

Téléchargements du document

17