From projection pursuit and CART to adaptive discriminant analysis?

Rémi Gribonval 1
1 METISS - Speech and sound data modeling and processing
IRISA - Institut de Recherche en Informatique et Systèmes Aléatoires, Inria Rennes – Bretagne Atlantique
Abstract : While many efforts have been put into the devel- opment of nonlinear approximation theory and its applications to signal and image compression, encoding and denoising, there seems to be very few theoretical developments of adaptive discriminant representations in the area of feature extraction, selection and signal classification. In this paper, we try to advocate the idea that such developments and efforts are worthwhile, based on the theorerical study of a data-driven discriminant analysis method on a simple---yet instructive---example. We consider the problem of classifying a signal drawn from a mixture of two classes, using its projections onto low-dimensional subspaces. Unlike the linear discriminant analysis (LDA) strategy, which selects subspaces that do not depend on the observed signal, we consider an adaptive sequential selection of projections, in the spirit of nonlinear approximation and classification and regression trees (CART): at each step, the subspace is enlarged in a direction that maximizes the mutual information with the unknown class. We derive explicit characterizations of this adaptive discriminant analysis (ADA) strategy in two situations. When the two classes are Gaussian with the same covariance matrix but different means, the adaptive subspaces are actually nonadaptive and can be computed with an algorithm similar to orthonormal matching pursuit. When the classes are centered Gaussians with different covariances, the adaptive subspaces are spanned by eigen-vectors of an operator given by the covariance matrices (just as could be predicted by regular LDA), however we prove that the order of observation of the components along these eigen-vectors actually depends on the observed signal. Numerical experiments on synthetic data illustrate how data-dependent features can be used to outperform LDA on a classification task, and we discuss how our results could be applied in practice.
Type de document :
Article dans une revue
Neural Networks, IEEE Transactions on, IEEE, 2005, 16 (3), pp.522--532. 〈10.1109/TNN.2005.844900〉
Liste complète des métadonnées

Littérature citée [27 références]  Voir  Masquer  Télécharger
Contributeur : Rémi Gribonval <>
Soumis le : mercredi 9 février 2011 - 07:58:41
Dernière modification le : vendredi 16 novembre 2018 - 01:22:48
Document(s) archivé(s) le : mardi 10 mai 2011 - 02:45:08


Accord explicite pour ce dépôt



Rémi Gribonval. From projection pursuit and CART to adaptive discriminant analysis?. Neural Networks, IEEE Transactions on, IEEE, 2005, 16 (3), pp.522--532. 〈10.1109/TNN.2005.844900〉. 〈inria-00564479〉



Consultations de la notice


Téléchargements de fichiers