Bayesian neural network unit priors and generalized Weibull-tail property - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Bayesian neural network unit priors and generalized Weibull-tail property

Résumé

The connection between Bayesian neural networks and Gaussian processes gained a lot of attention in the last few years. Hidden units are proven to follow a Gaussian process limit when the layer width tends to infinity. Recent work has suggested that finite Bayesian neural networks may outperform their infinite counterparts because they adapt their internal representations flexibly. To establish solid ground for future research on finite-width neural networks, our goal is to study the prior induced on hidden units. Our main result is an accurate description of hidden units tails which shows that unit priors become heavier-tailed going deeper, thanks to the introduced notion of generalized Weibull-tail. This finding sheds light on the behavior of hidden units of finite Bayesian neural networks.
Fichier principal
Vignette du fichier
ACML___Accurate_tails_of_BNN_unit_priors-3.pdf (767.1 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03368522 , version 1 (06-10-2021)

Identifiants

  • HAL Id : hal-03368522 , version 1

Citer

Mariia Vladimirova, Julyan Arbel, Stéphane Girard. Bayesian neural network unit priors and generalized Weibull-tail property. ACML 2021 - 13th Asian Conference on Machine Learning, Nov 2021, Virtual, Unknown Region. pp.1-16. ⟨hal-03368522⟩
67 Consultations
53 Téléchargements

Partager

Gmail Facebook X LinkedIn More