# A Large Dimensional Analysis of Least Squares Support Vector Machines

Abstract : In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data $p$ and their number $n$ are both large, that the LS-SVM decision function can be well approximated by a normally distributed random variable, the mean and variance of which depend explicitly on a local behavior of the kernel function. This theoretical result is then applied to the MNIST and Fashion-MNIST datasets which, despite their non-Gaussianity, exhibit a convincingly close behavior. Most importantly, our analysis provides a deeper understanding of the mechanism into play in SVM-type methods and in particular of the impact on the choice of the kernel function as well as some of their theoretical limits in separating high dimensional Gaussian vectors.
Keywords :
Document type :
Journal articles
Domain :

https://hal.inria.fr/hal-02048984
Contributor : Zhenyu Liao <>
Submitted on : Tuesday, February 26, 2019 - 8:43:27 AM
Last modification on : Tuesday, March 5, 2019 - 2:47:06 PM

### Citation

Zhenyu Liao, Romain Couillet. A Large Dimensional Analysis of Least Squares Support Vector Machines. IEEE Transactions on Signal Processing, Institute of Electrical and Electronics Engineers, 2019, 67 (4), pp.1065-1074. ⟨10.1109/TSP.2018.2889954⟩. ⟨hal-02048984⟩

Record views