Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method

Simon Lacoste-Julien 1, 2 Mark Schmidt 1, 2 Francis Bach 1, 2
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to existing techniques, with similar performance behavior.
Complete list of metadata

https://hal.inria.fr/hal-00768187
Contributor : Simon Lacoste-Julien <>
Submitted on : Friday, December 21, 2012 - 2:09:48 AM
Last modification on : Thursday, July 1, 2021 - 5:58:07 PM

Links full text

Identifiers

  • HAL Id : hal-00768187, version 1
  • ARXIV : 1212.2002

Collections

Citation

Simon Lacoste-Julien, Mark Schmidt, Francis Bach. A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method. 2012. ⟨hal-00768187⟩

Share

Metrics

Record views

353