Large-Scale Machine Learning and Applications

Julien Mairal 1
1 Thoth - Apprentissage de modèles à partir de données massives
LJK - Laboratoire Jean Kuntzmann, Inria Grenoble - Rhône-Alpes
Abstract : This thesis presents my main research activities in statistical machine learning after my PhD, starting from my post-doc at UC Berkeley to my present research position at Inria Grenoble. The first chapter introduces the context and a summary of my scientific contributions and emphasizes the importance of pluri-disciplinary research. For instance, mathematical optimization has become central in machine learning and the interplay between signal processing, statistics, bioinformatics, and computer vision is stronger than ever. With many scientific and industrial fields producing massive amounts of data, the impact of machine learning is potentially huge and diverse. However, dealing with massive data raises also many challenges. In this context, the manuscript presents different contributions, which are organized in three main topics. Chapter 2 is devoted to large-scale optimization in machine learning with a focus on algorithmic methods. We start with majorization-minimization algorithms for structured problems, including block-coordinate, incremental, and stochastic variants. These algorithms are analyzed in terms of convergence rates for convex problems and in terms of convergence to stationary points for non-convex ones. We also introduce fast schemes for minimizing large sums of convex functions and principles to accelerate gradient-based approaches, based on Nesterov’s acceleration and on Quasi-Newton approaches. Chapter 3 presents the paradigm of deep kernel machine, which is an alliance between kernel methods and multilayer neural networks. In the context of visual recognition, we introduce a new invariant image model called convolutional kernel networks, which is a new type of convolutional neural network with a reproducing kernel interpretation. The network comes with simple and effective principles to do unsupervised learning, and is compatible with supervised learning via backpropagation rules. Chapter 4 is devoted to sparse estimation—that is, the automatic selection of model variables for explaining observed data; in particular, this chapter presents the result of pluri-disciplinary collaborations in bioinformatics and neuroscience where the sparsity principle is a key to build intepretable predictive models. Finally, the last chapter concludes the manuscript and suggests future perspectives.
Liste complète des métadonnées

Cited literature [243 references]  Display  Hide  Download
Contributor : Julien Mairal <>
Submitted on : Tuesday, November 7, 2017 - 9:57:55 AM
Last modification on : Wednesday, April 11, 2018 - 1:58:20 AM
Document(s) archivé(s) le : Thursday, February 8, 2018 - 12:36:46 PM


Files produced by the author(s)


  • HAL Id : tel-01629997, version 1



Julien Mairal. Large-Scale Machine Learning and Applications. Machine Learning [stat.ML]. UGA - Université Grenoble Alpes, 2017. ⟨tel-01629997⟩



Record views


Files downloads