Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks

Résumé

The lottery ticket hypothesis states that a randomly-initialized neural network contains a small subnetwork which, when trained in isolation, can compete with the performance of the original network. Recent theoretical works proved an even stronger version: every sufficiently overparameterized (dense) neural network contains a subnetwork that, even without training, achieves accuracy comparable to that of the trained large network. These works left as an open problem to extend the result to convolutional neural networks (CNNs). In this work we provide such generalization by showing that, with high probability, it is possible to approximate any CNN by pruning a random CNN whose size is larger by a logarithmic factor.
Fichier principal
Vignette du fichier
SLTHforCNNs.pdf (342.97 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03548226 , version 1 (29-01-2022)

Identifiants

  • HAL Id : hal-03548226 , version 1

Citer

Arthur da Cunha, Emanuele Natale, Laurent Viennot. Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks. ICLR 2022 - 10th International Conference on Learning Representations, Apr 2022, Virtual, France. ⟨hal-03548226⟩
481 Consultations
744 Téléchargements

Partager

Gmail Facebook X LinkedIn More