Skip to Main content Skip to Navigation
Theses

Structural Learning of Neural Networks

Pierre Wolinski 1
1 TAU - TAckling the Underspecified
Inria Saclay - Ile de France, LRI - Laboratoire de Recherche en Informatique
Abstract : The structure of a neural network determines to a large extent its cost of training and use, as well as its ability to learn. These two aspects are usually in competition: the larger a neural network is, the better it will perform the task assigned to it, but the more it will require memory and computing time resources for training. Automating the search of efficient network structures -of reasonable size and performing well- is then a very studied question in this area. Within this context, neural networks with various structures are trained, which requires a new set of training hyperparameters for each new structure tested. The aim of the thesis is to address different aspects of this problem. The first contribution is a training method that operates within a large perimeter of network structures and tasks, without needing to adjust the learning rate. The second contribution is a network training and pruning technique, designed to be insensitive to the initial width of the network. The last contribution is mainly a theorem that makes possible to translate an empirical training penalty into a Bayesian prior, theoretically well founded. This work results from a search for properties that theoretically must be verified by training and pruning algorithms to be valid over a wide range of neural networks and objectives.
Complete list of metadatas

Cited literature [116 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02888604
Contributor : Abes Star :  Contact
Submitted on : Friday, July 3, 2020 - 10:26:17 AM
Last modification on : Saturday, October 10, 2020 - 3:34:47 AM
Long-term archiving on: : Thursday, September 24, 2020 - 6:55:02 AM

File

81525_WOLINSKI_2020_archivage....
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02888604, version 1

Citation

Pierre Wolinski. Structural Learning of Neural Networks. Neural and Evolutionary Computing [cs.NE]. Université Paris-Saclay, 2020. English. ⟨NNT : 2020UPASS026⟩. ⟨tel-02888604⟩

Share

Metrics

Record views

143

Files downloads

106