Faster ICA by preconditioning with Hessian approximations

Abstract : Independent Component Analysis (ICA) is a powerful technique for unsupervised data exploration that is widely used across fields such as neuroscience, astronomy, chemistry or biology. Linear ICA is a linear latent factor model, similar to sparse dictionary learning, that aims at discovering statistically independent sources from multivariate observations. ICA is a probabilistic generative model for which inference is classically done by maximum likelihood estimation. Estimating sources by maximum likelihood leads to a smooth non-convex optimization problem where the unknown is a matrix called the separating or unmixing matrix. As the gradient of the likelihood is available in closed form, first order gradient methods, stochastic or non-stochastic, are often employed despite a slow convergence such as in the Infomax algorithm. While the Hessian is known analytically, the cost of its computation and inversion makes Newton method unpractical for a large number of sources. We show how sparse and positive approximations of the true Hessian can be obtained and used to precondition the L-BFGS algorithm. Results on simulations and two applied problems (EEG data and image patches) demonstrate that the proposed technique leads to convergence that can be orders of magnitude faster than algorithms commonly used today even when looking for hundred of sources.
Type de document :
Pré-publication, Document de travail
Liste complète des métadonnées

Littérature citée [45 références]  Voir  Masquer  Télécharger
Contributeur : Pierre Ablin <>
Soumis le : lundi 3 juillet 2017 - 10:48:30
Dernière modification le : jeudi 11 janvier 2018 - 06:27:34
Document(s) archivé(s) le : jeudi 14 décembre 2017 - 18:36:06


quasi-newton-methods (6).pdf
Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-01552340, version 1
  • ARXIV : 1706.08171


Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort. Faster ICA by preconditioning with Hessian approximations. 2017. 〈hal-01552340v1〉



Consultations de la notice


Téléchargements de fichiers