Learning Sparse Features with an Auto-Associator

Sébastien Rebecchi 1 Hélène Paugam-Moisy 2 Michèle Sebag 3, 1
1 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
2 DM2L - Data Mining and Machine Learning
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : A major issue in statistical machine learning is the design of a representa-tion, or feature space, facilitating the resolution of the learning task at hand. Sparse representations in particular facilitate discriminant learning: On the one hand, they are robust to noise. On the other hand, they disentangle the factors of variation mixed up in dense representations, favoring the separa-bility and interpretation of data. This chapter focuses on auto-associators (AAs), i.e. multi-layer neural networks trained to encode/decode the data and thus de facto defining a feature space. AAs, first investigated in the 80s, were recently reconsidered as building blocks for deep neural networks. This chapter surveys related work about building sparse representations, and presents a new non-linear explicit sparse representation method referred to as Sparse Auto-Associator (SAA), integrating a sparsity objective within the standard auto-associator learning criterion. The comparative empirical val-idation of SAAs on state-of-art handwritten digit recognition benchmarks shows that SAAs outperform standard auto-associators in terms of classifi-cation performance and yield similar results as denoising auto-associators. Furthermore, SAAs enable to control the representation size to some extent, through a conservative pruning of the feature space.
Type de document :
Chapitre d'ouvrage
Kowaliw, Taras and Bredeche, Nicolas and Doursat, René. Growing Adaptive Machines, 557, Springer Verlag, pp.139 - 158, 2014, Studies in Computational Intelligence, 〈10.1007/978-3-642-55337-0_4〉
Liste complète des métadonnées

Littérature citée [32 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01109773
Contributeur : Marc Schoenauer <>
Soumis le : lundi 26 janvier 2015 - 23:52:00
Dernière modification le : mercredi 31 octobre 2018 - 12:24:25
Document(s) archivé(s) le : samedi 12 septembre 2015 - 06:35:09

Fichier

chapterSAA-Paugam.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Sébastien Rebecchi, Hélène Paugam-Moisy, Michèle Sebag. Learning Sparse Features with an Auto-Associator. Kowaliw, Taras and Bredeche, Nicolas and Doursat, René. Growing Adaptive Machines, 557, Springer Verlag, pp.139 - 158, 2014, Studies in Computational Intelligence, 〈10.1007/978-3-642-55337-0_4〉. 〈hal-01109773〉

Partager

Métriques

Consultations de la notice

500

Téléchargements de fichiers

355