Lifted coordinate descent for learning with trace-norm regularization

Miro Dudik 1 Zaid Harchaoui 2 Jérôme Malick 3
1 NY
Yahoo! Labs New York
2 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
3 BIPOP - Modelling, Simulation, Control and Optimization of Non-Smooth Dynamical Systems
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We consider the minimization of a smooth loss with trace-norm regularization, which is a natural objective in multi-class and multi-task learning. Even though the problem is convex, existing approaches rely on optimizing a non-convex variational bound, which is not guaranteed to converge, or repeatedly perform singular-value decomposition, which prevents scaling beyond moderate matrix sizes. We lift the non-smooth convex problem into an infinitely dimensional smooth problem and apply coordinate descent to solve it. We prove that our approach converges to the optimum, and is competitive or outperforms state of the art.
Type de document :
Communication dans un congrès
AISTATS - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics - 2012, Apr 2012, La Palma, Spain. 22, pp.327-336, 2012, JMLR Workshop and Conference Proceedings
Liste complète des métadonnées

Littérature citée [39 références]  Voir  Masquer  Télécharger


https://hal.inria.fr/hal-00756802
Contributeur : Thoth Team <>
Soumis le : vendredi 23 novembre 2012 - 17:24:54
Dernière modification le : jeudi 11 janvier 2018 - 06:21:56
Document(s) archivé(s) le : dimanche 24 février 2013 - 03:55:10

Fichiers

dhm_2012_rod_atomdescent.pdf
Accord explicite pour ce dépôt

Identifiants

  • HAL Id : hal-00756802, version 1

Citation

Miro Dudik, Zaid Harchaoui, Jérôme Malick. Lifted coordinate descent for learning with trace-norm regularization. AISTATS - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics - 2012, Apr 2012, La Palma, Spain. 22, pp.327-336, 2012, JMLR Workshop and Conference Proceedings. 〈hal-00756802〉

Partager

Métriques

Consultations de la notice

942

Téléchargements de fichiers

519