Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Optimization of deep neural networks: a survey and unified taxonomy

El-Ghazali Talbi 1, 2, 3
1 BONUS - Optimisation de grande taille et calcul large échelle
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : During the last years, research in applying optimization approaches in the automatic design of deep neural networks (DNNs) becomes increasingly popular. Although various appproaches have been proposed, there is a lack of a comprehensive survey and taxonomy on this hot research topic. In this paper, we propose a unified way to describe the various optimization algorithms which focus on common and important search components of optimization algorithms: representation, objective function, constraints, initial solution(s) and variation operators. In addition to large scale search space, the problem is characterized by its variable mixed design space, very expensive and multiple blackbox objective functions. Hence, this unified methodology has been extended to advanced optimization approaches such as surrogate-based, multi-objective and parallel optimization.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [206 references]  Display  Hide  Download

https://hal.inria.fr/hal-02570804
Contributor : Talbi El-Ghazali <>
Submitted on : Wednesday, June 3, 2020 - 11:25:57 AM
Last modification on : Thursday, June 4, 2020 - 3:50:02 AM

File

Opti-ML.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02570804, version 2

Collections

Citation

El-Ghazali Talbi. Optimization of deep neural networks: a survey and unified taxonomy. 2020. ⟨hal-02570804v2⟩

Share

Metrics

Record views

138

Files downloads

640