| HAL-Inria | Publications, software ... of Inria's scientists |
Conference papers
Online and Stochastic Optimization beyond Lipschitz Continuity: A Riemannian Approach
Abstract : Motivated by applications to machine learning and imaging science, we study a class of online and stochastic optimization problems with loss functions that are not Lipschitz continuous; in particular, the loss functions encountered by the optimizer could exhibit gradient singularities or be singular themselves. Drawing on tools and techniques from Riemannian geometry, we examine a Riemann-Lipschitz (RL) continuity condition which is tailored to the singularity landscape of the problem's loss functions. In this way, we are able to tackle cases beyond the Lipschitz framework provided by a global norm, and we derive optimal regret bounds and last iterate convergence results through the use of regularized learning methods (such as online mirror descent). These results are subsequently validated in a class of stochastic Poisson inverse problems that arise in imaging science.
https://hal.inria.fr/hal-03043671
Contributor : Panayotis Mertikopoulos Connect in order to contact the contributor
Submitted on : Monday, December 7, 2020 - 1:43:16 PM
Last modification on : Wednesday, November 3, 2021 - 7:58:03 AM
Long-term archiving on: : Monday, March 8, 2021 - 7:03:27 PM
Contributor : Panayotis Mertikopoulos Connect in order to contact the contributor
Submitted on : Monday, December 7, 2020 - 1:43:16 PM
Last modification on : Wednesday, November 3, 2021 - 7:58:03 AM
Long-term archiving on: : Monday, March 8, 2021 - 7:03:27 PM
File
RiemannLipschitz-ICLR.pdf
Files produced by the author(s)