A Large Dimensional Analysis of Least Squares Support Vector Machines - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Signal Processing Année : 2019

A Large Dimensional Analysis of Least Squares Support Vector Machines

Zhenyu Liao
Romain Couillet

Résumé

In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data $p$ and their number $n$ are both large, that the LS-SVM decision function can be well approximated by a normally distributed random variable, the mean and variance of which depend explicitly on a local behavior of the kernel function. This theoretical result is then applied to the MNIST and Fashion-MNIST datasets which, despite their non-Gaussianity, exhibit a convincingly close behavior. Most importantly, our analysis provides a deeper understanding of the mechanism into play in SVM-type methods and in particular of the impact on the choice of the kernel function as well as some of their theoretical limits in separating high dimensional Gaussian vectors.
Fichier principal
Vignette du fichier
couillet_squares.pdf (375.19 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02048984 , version 1 (19-05-2020)

Identifiants

Citer

Zhenyu Liao, Romain Couillet. A Large Dimensional Analysis of Least Squares Support Vector Machines. IEEE Transactions on Signal Processing, 2019, 67 (4), pp.1065-1074. ⟨10.1109/TSP.2018.2889954⟩. ⟨hal-02048984⟩
97 Consultations
194 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More