Online Learning for Matrix Factorization and Sparse Coding

Julien Mairal 1, 2, * Francis Bach 1, 2 Jean Ponce 1, 2 Guillermo Sapiro 3
* Auteur correspondant
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : Sparse coding---that is, modelling data vectors as sparse linear combinations of basis elements---is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set, adapting it to specific data. Variations of this problem include dictionary learning in signal processing, non-negative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large datasets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to state-of-the-art performance in terms of speed and optimization for both small and large datasets.
Liste complète des métadonnées

Littérature citée [85 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00408716
Contributeur : Julien Mairal <>
Soumis le : mercredi 10 février 2010 - 20:18:13
Dernière modification le : mardi 24 avril 2018 - 17:20:13
Document(s) archivé(s) le : jeudi 23 septembre 2010 - 11:18:04

Fichiers

arxiv_mairal10a.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00408716, version 2
  • ARXIV : 0908.0050

Collections

Citation

Julien Mairal, Francis Bach, Jean Ponce, Guillermo Sapiro. Online Learning for Matrix Factorization and Sparse Coding. Journal of Machine Learning Research, Journal of Machine Learning Research, 2010, 11 (1), pp.19--60. 〈http://jmlr.csail.mit.edu/papers/volume11/mairal10a/mairal10a.pdf〉. 〈inria-00408716v2〉

Partager

Métriques

Consultations de la notice

3578

Téléchargements de fichiers

547