Optimization with First-Order Surrogate Functions

Julien Mairal 1, *
* Corresponding author
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as accelerated proximal gradient, block coordinate descent, or Frank-Wolfe algorithms. Second, we introduce a new incremental scheme that experimentally matches or outperforms state-of-the-art solvers for large-scale optimization problems typically arising in machine learning.
Complete list of metadatas

https://hal.inria.fr/hal-00822229
Contributor : Julien Mairal <>
Submitted on : Tuesday, May 14, 2013 - 12:28:06 PM
Last modification on : Monday, December 17, 2018 - 11:22:02 AM
Long-term archiving on : Thursday, August 15, 2013 - 4:15:18 AM

Files

main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00822229, version 1
  • ARXIV : 1305.3120

Collections

Citation

Julien Mairal. Optimization with First-Order Surrogate Functions. ICML 2013 - International Conference on Machine Learning, Jun 2013, Atlanta, United States. pp.783-791. ⟨hal-00822229⟩

Share

Metrics

Record views

5060

Files downloads

2527