Online and Stochastic Optimization beyond Lipschitz Continuity: A Riemannian Approach - Archive ouverte HAL Access content directly
Conference Papers Year :

Online and Stochastic Optimization beyond Lipschitz Continuity: A Riemannian Approach

(1) , (2) , (1)
1
2

Abstract

Motivated by applications to machine learning and imaging science, we study a class of online and stochastic optimization problems with loss functions that are not Lipschitz continuous; in particular, the loss functions encountered by the optimizer could exhibit gradient singularities or be singular themselves. Drawing on tools and techniques from Riemannian geometry, we examine a Riemann-Lipschitz (RL) continuity condition which is tailored to the singularity landscape of the problem's loss functions. In this way, we are able to tackle cases beyond the Lipschitz framework provided by a global norm, and we derive optimal regret bounds and last iterate convergence results through the use of regularized learning methods (such as online mirror descent). These results are subsequently validated in a class of stochastic Poisson inverse problems that arise in imaging science.
Fichier principal
Vignette du fichier
RiemannLipschitz-ICLR.pdf (3.08 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03043671 , version 1 (07-12-2020)

Identifiers

  • HAL Id : hal-03043671 , version 1

Cite

Kimon Antonakopoulos, Elena Veronica Belmega, Panayotis Mertikopoulos. Online and Stochastic Optimization beyond Lipschitz Continuity: A Riemannian Approach. ICLR 2020 - International Conference on Learning Representations, Apr 2020, Addis Ababa, Ethiopia. pp.1-20. ⟨hal-03043671⟩
152 View
93 Download

Share

Gmail Facebook Twitter LinkedIn More