Skip to Main content Skip to Navigation
New interface

Structural Learning of Neural Networks

Pierre Wolinski 1, 2 
2 TAU - TAckling the Underspecified
Inria Saclay - Ile de France, LRI - Laboratoire de Recherche en Informatique
Abstract : The structure of a neural network determines to a large extent its cost of training and use, as well as its ability to learn. These two aspects are usually in competition: the larger a neural network is, the better it will perform the task assigned to it, but the more it will require memory and computing time resources for training. Automating the search of efficient network structures -of reasonable size and performing well- is then a very studied question in this area. Within this context, neural networks with various structures are trained, which requires a new set of training hyperparameters for each new structure tested. The aim of the thesis is to address different aspects of this problem. The first contribution is a training method that operates within a large perimeter of network structures and tasks, without needing to adjust the learning rate. The second contribution is a network training and pruning technique, designed to be insensitive to the initial width of the network. The last contribution is mainly a theorem that makes possible to translate an empirical training penalty into a Bayesian prior, theoretically well founded. This work results from a search for properties that theoretically must be verified by training and pruning algorithms to be valid over a wide range of neural networks and objectives.
Complete list of metadata

Cited literature [116 references]  Display  Hide  Download
Contributor : ABES STAR :  Contact
Submitted on : Friday, July 3, 2020 - 10:26:17 AM
Last modification on : Sunday, June 26, 2022 - 2:51:35 AM
Long-term archiving on: : Thursday, September 24, 2020 - 6:55:02 AM


Version validated by the jury (STAR)


  • HAL Id : tel-02888604, version 1


Pierre Wolinski. Structural Learning of Neural Networks. Neural and Evolutionary Computing [cs.NE]. Université Paris-Saclay, 2020. English. ⟨NNT : 2020UPASS026⟩. ⟨tel-02888604⟩



Record views


Files downloads