Convex relaxations of penalties for sparse correlated variables with bounded total variation

Abstract : We study the problem of statistical estimation with a signal known to be sparse, spatially contiguous, and containing many highly correlated variables. We take inspiration from the recently introduced k-support norm, which has been successfully applied to sparse prediction problems with correlated features, but lacks any explicit structural constraints commonly found in machine learning and image processing. We address this problem by incorporating a total variation penalty in the k-support framework. We introduce the (k, s) support total variation norm as the tightest convex relaxation of the intersection of a set of sparsity and total variation constraints. We show that this norm leads to an intractable combinatorial graph optimization problem, which we prove to be NP-hard. We then introduce a tractable relaxation with approximation guarantees that scale well for grid structured graphs. We devise several first-order optimization strategies for statistical parameter estimation with the described penalty. We demonstrate the effectiveness of this penalty on classification in the low-sample regime, classification with M/EEG neuroimaging data, and image recovery with synthetic and real data background subtracted image recovery tasks. We extensively analyse the application of our penalty on the complex task of identifying predictive regions from low-sample high-dimensional fMRI brain data, we show that our method is particularly useful compared to existing methods in terms of accuracy, interpretability, and stability.
Type de document :
Article dans une revue
Machine Learning, Springer Verlag, 2015, pp.1-21. 〈10.1007/s10994-015-5511-2〉
Liste complète des métadonnées

Littérature citée [24 références]  Voir  Masquer  Télécharger
Contributeur : Matthew Blaschko <>
Soumis le : mercredi 15 juillet 2015 - 16:49:52
Dernière modification le : vendredi 12 janvier 2018 - 11:15:39
Document(s) archivé(s) le : mercredi 26 avril 2017 - 05:41:27


Fichiers produits par l'(les) auteur(s)



Eugene Belilovsky, Andreas Argyriou, Gaël Varoquaux, Matthew Blaschko. Convex relaxations of penalties for sparse correlated variables with bounded total variation. Machine Learning, Springer Verlag, 2015, pp.1-21. 〈10.1007/s10994-015-5511-2〉. 〈hal-01167861〉



Consultations de la notice


Téléchargements de fichiers