inria-00174298, version 1

Resampling methods for parameter-free and robust feature selection with mutual information

Damien François () ¹, Fabrice Rossi () ², Vincent Wertz () ¹, Michel Verleysen ³

Neurocomputing 70, 7-9 (2007) 1276-1288

• 1:  Centre for Systems Engineering and Applied Mechanics (CSAM)
• http://www.uclouvain.be/en-csam.html
  Université Catholique de Louvain (UCL) - Belgique Bâtiment Euler, 4-6 avenue G. Lemaître B-1348 Louvain-la-Neuve (Belgique) Belgium
• 2:  AxIS (INRIA Rocquencourt / INRIA Sophia Antipolis)
• 
  INRIA France
• 3:  Dispositifs Intégrés et Circuits Electroniques Machine Learning Group (DICE - MLG)
• http://www.dice.ucl.ac.be/
  Université Catholique de Louvain (UCL) - Belgique Université catholique de Louvain Engineering Faculty - Electricity Department DICE - Microelectronics laboratory 3 place du Levant B-1348 Louvain-la-Neuve Belgium

Bibliographic reference

• Type of document: Articles in peer-reviewed journal
• Domain:

  Computer Science/Neural and Evolutionary Computing

  Computer Science/Learning

  Mathematics/Statistics

• Title: Resampling methods for parameter-free and robust feature selection with mutual information
• Abstract: Combining the mutual information criterion with a forward feature selection strategy offers a good trade-off between optimality of the selected feature subset and computation time. However, it requires to set the parameter(s) of the mutual information estimator and to determine when to halt the forward procedure. These two choices are difficult to make because, as the dimensionality of the subset increases, the estimation of the mutual information becomes less and less reliable. This paper proposes to use resampling methods, a K-fold cross-validation and the permutation test, to address both issues. The resampling methods bring information about the variance of the estimator, information which can then be used to automatically set the parameter and to calculate a threshold to stop the forward procedure. The procedure is illustrated on a synthetic dataset as well as on real-world examples.
• Full text language: English
• Journal title:

  Neurocomputing
  Publisher	Elsevier
  ISSN	0925-2312

• Publication date: 2007
• Audience: international
• Commercial editor: Elsevier
• Volume: 70
• Number: 7-9
• Pagination: 1276-1288
• DOI: 10.1016/j.neucom.2006.11.019
• Keywords: Mutual information – Permutation test – Feature selection – Resampling

Attached file list to this document: 

TEX

hal

illustratechoiceofk.eps(12.1 KB)

forward-delve-1.eps(10.8 KB)

friedman-forward.eps(31.2 KB)

forwardnitro.eps(12.5 KB)

elsart.cls(53.1 KB)

permtest.bbl(6.8 KB)

PermutationTest.bib(63.5 KB)

forwardtoy.eps(10.1 KB)

permtest.tex(53.5 KB)

friedman-mi-v4v10-quantile.eps(5.1 KB)

forwardhousing.eps(10.2 KB)

elsart-num.bst(23.4 KB)

illustratechoiceofk2.eps(55.2 KB)

PS

permtest.ps(323.3 KB)

PDF

permtest.pdf(240 KB)

• inria-00174298, version 1
• http://hal.inria.fr/inria-00174298
• oai:hal.inria.fr:inria-00174298
• From: Fabrice Rossi <>
• Submitted on: Sunday, 23 September 2007 15:35:36
• Updated on: Sunday, 23 September 2007 16:09:36