Leveraging k-NN for generic classification boosting

Abstract : Voting rules relying on k-nearest neighbors (k-NN) are an effective tool in countless many machine learning techniques. Thanks to its simplicity, k-NN classification is very attractive to practitioners, as it enables very good performances in several practical applications. However, it suffers from various drawbacks, like sensitivity to "noisy" instances and poor generalization properties when dealing with sparse high-dimensional data. In this paper, we tackle the k-NN classification problem at its core by providing a novel k-NN boosting approach. Namely, we propose a supervised learning algorithm, called Universal Nearest Neighbors (UNN), that induces a leveraged k-NN rule by globally minimizing a surrogate risk upper bounding the empirical misclassification rate over training data. Interestingly, this surrogate risk can be arbitrary chosen from a class of Bregman loss functions, including the familiar exponential, logistic and squared losses. Furthermore, we show that UNN allows to efficiently filter a dataset of instances by keeping only a small fraction of data. Experimental results on the synthetic Ripley's dataset show that such a filtering strategy is able to reject "noisy" examples, and yields a classification error close to the optimal Bayes error. Experiments on standard UCI datasets show significant improvements over the current state of the art.
Type de document :
Article dans une revue
Liste complète des métadonnées

https://hal.inria.fr/hal-00664462
Contributeur : Michel Barlaud <>
Soumis le : lundi 30 janvier 2012 - 16:17:37
Dernière modification le : dimanche 6 décembre 2015 - 01:03:36

Identifiants

  • HAL Id : hal-00664462, version 1

Collections

I3S | UNICE | BNRMI | GAEL | UGA

Citation

Paolo Piro, Richard Nock, Frank Nielsen, Michel Barlaud. Leveraging k-NN for generic classification boosting. Neurocomputing, Elsevier, 2012, 80, pp.3-9. <http://www.sciencedirect.com/science/article/pii/S0925231211005984>. <hal-00664462>

Partager

Métriques

Consultations de la notice

522