Optimization of deep neural networks: a survey and unified taxonomy - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Optimization of deep neural networks: a survey and unified taxonomy

Optimisation de réseaux de neurones profonds: une taxinomie unifiée

Résumé

During the last years, research in applying optimization approaches in the automatic design of deep neural networks (DNNs) becomes increasingly popular. Although various appproaches have been proposed, there is a lack of a comprehensive survey and taxonomy on this hot research topic. In this paper, we propose a unified way to describe the various optimization algorithms which focus on common and important search components of optimization algorithms: representation, objective function, constraints, initial solution(s) and variation operators. In addition to large scale search space, the problem is characterized by its variable mixed design space, very expensive and multiple blackbox objective functions. Hence, this unified methodology has been extended to advanced optimization approaches such as surrogate-based, multi-objective and parallel optimization.
Fichier principal
Vignette du fichier
Opti-ML.pdf (1.02 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02570804 , version 1 (12-05-2020)
hal-02570804 , version 2 (03-06-2020)

Identifiants

  • HAL Id : hal-02570804 , version 1

Citer

El-Ghazali Talbi. Optimization of deep neural networks: a survey and unified taxonomy. 2020. ⟨hal-02570804v1⟩
1370 Consultations
3646 Téléchargements

Partager

Gmail Facebook X LinkedIn More