Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Journal of Machine Learning Research Année : 2019

Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations

Résumé

The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional representations of signals; we study their invariance to translations and to more general groups of transformations, their stability to the action of diffeomorphisms, and their ability to preserve signal information. This analysis is carried by introducing a multilayer kernel based on convolutional kernel networks and by studying the geometry induced by the kernel mapping. We then characterize the corresponding reproducing kernel Hilbert space (RKHS), showing that it contains a large class of convolutional neural networks with homogeneous activation functions. This analysis allows us to separate data representation from learning, and to provide a canonical measure of model complexity, the RKHS norm, which controls both stability and generalization of any learned model. In addition to models in the constructed RKHS, our stability analysis also applies to convolutional networks with generic activations such as rectified linear units, and we discuss its relationship with recent generalization bounds based on spectral norms.
Fichier principal
Vignette du fichier
main.pdf (564.6 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01536004 , version 1 (09-06-2017)
hal-01536004 , version 2 (14-11-2017)
hal-01536004 , version 3 (08-04-2018)
hal-01536004 , version 4 (10-10-2018)

Identifiants

Citer

Alberto Bietti, Julien Mairal. Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations. Journal of Machine Learning Research, 2019, 20 (1), pp.1-49. ⟨hal-01536004v4⟩
1100 Consultations
938 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More