Skip to Main content Skip to Navigation
Journal articles

Convex relaxations of penalties for sparse correlated variables with bounded total variation

Abstract : We study the problem of statistical estimation with a signal known to be sparse, spatially contiguous, and containing many highly correlated variables. We take inspiration from the recently introduced k-support norm, which has been successfully applied to sparse prediction problems with correlated features, but lacks any explicit structural constraints commonly found in machine learning and image processing. We address this problem by incorporating a total variation penalty in the k-support framework. We introduce the (k, s) support total variation norm as the tightest convex relaxation of the intersection of a set of sparsity and total variation constraints. We show that this norm leads to an intractable combinatorial graph optimization problem, which we prove to be NP-hard. We then introduce a tractable relaxation with approximation guarantees that scale well for grid structured graphs. We devise several first-order optimization strategies for statistical parameter estimation with the described penalty. We demonstrate the effectiveness of this penalty on classification in the low-sample regime, classification with M/EEG neuroimaging data, and image recovery with synthetic and real data background subtracted image recovery tasks. We extensively analyse the application of our penalty on the complex task of identifying predictive regions from low-sample high-dimensional fMRI brain data, we show that our method is particularly useful compared to existing methods in terms of accuracy, interpretability, and stability.
Document type :
Journal articles
Complete list of metadata

Cited literature [24 references]  Display  Hide  Download
Contributor : Matthew Blaschko Connect in order to contact the contributor
Submitted on : Wednesday, July 15, 2015 - 4:49:52 PM
Last modification on : Monday, December 13, 2021 - 9:16:01 AM
Long-term archiving on: : Wednesday, April 26, 2017 - 5:41:27 AM


Files produced by the author(s)



Eugene Belilovsky, Andreas Argyriou, Gaël Varoquaux, Matthew Blaschko. Convex relaxations of penalties for sparse correlated variables with bounded total variation. Machine Learning, Springer Verlag, 2015, pp.1-21. ⟨10.1007/s10994-015-5511-2⟩. ⟨hal-01167861⟩



Les métriques sont temporairement indisponibles