A sparse version of the ridge logistic regression for large-scale text categorization

Sujeevan Aseervatham; Anestis Antoniadis; Éric Gaussier; Michel Burlet; Yves Denneulin

hal-00633629, version 1

A sparse version of the ridge logistic regression for large-scale text categorization

Sujeevan Aseervatham () ¹, Anestis Antoniadis () ², Éric Gaussier ( Auteur à contacter de préférence ) ³, Michel Burlet () ^4, 5, Yves Denneulin () ⁶

Pattern Recognition Letters 32, 2 (2011) 101-106

Résumé : The ridge logistic regression has successfully been used in text categorization problems and it has been shown to reach the same performance as the Support Vector Machine but with the main advantage of computing a probability value rather than a score. However, the dense solution of the ridge makes its use unpractical for large scale categorization. On the other side, LASSO regularization is able to produce sparse solutions but its performance is dominated by the ridge when the number of features is larger than the number of observations and/or when the features are highly correlated. In this paper, we propose a new model selection method which tries to approach the ridge solution by a sparse solution. The method first computes the ridge solution and then performs feature selection. The experimental evaluations show that our method gives a solution which is a good trade-off between the ridge and LASSO solutions.

1 : Laboratoire d'informatique de Paris-nord (LIPN)
CNRS : UMR7030 – Université Paris XIII - Paris Nord
2 : Laboratoire Jean Kuntzmann (LJK)
CNRS : UMR5224 – Université Joseph Fourier - Grenoble I – Université Pierre-Mendès-France - Grenoble II – Institut Polytechnique de Grenoble - Grenoble Institute of Technology
3 : Laboratoire d'Informatique de Grenoble (LIG)
Université Joseph Fourier - Grenoble I – Institut Polytechnique de Grenoble - Grenoble Institute of Technology – Université Pierre-Mendès-France - Grenoble II – CNRS : UMR5217
4 : Laboratoire Leibniz (Leibniz - IMAG)
CNRS : UMR5522 – Université Joseph Fourier - Grenoble I – Institut National Polytechnique de Grenoble (INPG)
5 : Laboratoire des sciences pour la conception, l'optimisation et la production (G-SCOP)
CNRS : UMR5272 – Institut National Polytechnique de Grenoble (INPG) – Université Joseph Fourier - Grenoble I
6 : MESCAL (INRIA Grenoble Rhône-Alpes / LIG laboratoire d'Informatique de Grenoble)
CNRS : UMR5217 – INRIA – Université Joseph Fourier - Grenoble I – Université Pierre-Mendès-France - Grenoble II – Institut polytechnique de Grenoble (Grenoble INP)

hal-00633629, version 1
http://hal.archives-ouvertes.fr/hal-00633629
oai:hal.archives-ouvertes.fr:hal-00633629
Contributeur : Marie Josèphe Perruet <>
Soumis le : Lundi 15 Octobre 2012, 18:21:33
Dernière modification le : Mercredi 19 Décembre 2012, 14:26:43

Voir la fiche détaillée

Documents associés

PDF :

DOI : 10.1016/j.patrec.2010.09.023

Exporter

Bibtex EndNote TEI RefWorks

%% hal-00633629, version 1
%% http://hal.archives-ouvertes.fr/hal-00633629
@article{aseervatham:hal-00633629,
    hal_id = {hal-00633629},
    url = {http://hal.archives-ouvertes.fr/hal-00633629},
    title = {{A sparse version of the ridge logistic regression for large-scale text categorization}},
    author = {Aseervatham, Sujeevan and Antoniadis, Anestis and Gaussier, {\'E}ric and Burlet, Michel and Denneulin, Yves},
    abstract = {{The ridge logistic regression has successfully been used in text categorization problems and it has been shown to reach the same performance as the Support Vector Machine but with the main advantage of computing a probability value rather than a score. However, the dense solution of the ridge makes its use unpractical for large scale categorization. On the other side, LASSO regularization is able to produce sparse solutions but its performance is dominated by the ridge when the number of features is larger than the number of observations and/or when the features are highly correlated. In this paper, we propose a new model selection method which tries to approach the ridge solution by a sparse solution. The method first computes the ridge solution and then performs feature selection. The experimental evaluations show that our method gives a solution which is a good trade-off between the ridge and LASSO solutions.}},
    keywords = {Logistic regression; Model selection; Text categorization; Large scale categorization},
    language = {Anglais},
    affiliation = {Laboratoire d'informatique de Paris-nord - LIPN , Laboratoire Jean Kuntzmann - LJK , Laboratoire d'Informatique de Grenoble - LIG , Laboratoire Leibniz - Leibniz - IMAG , Laboratoire des sciences pour la conception, l'optimisation et la production - G-SCOP , MESCAL - INRIA Grenoble Rh{\^o}ne-Alpes / LIG laboratoire d'Informatique de Grenoble},
    pages = {101-106},
    journal = {Pattern Recognition Letters},
    volume = {32},
    number = {2 },
    note = {OSP OSP },
    audience = {internationale },
    doi = {10.1016/j.patrec.2010.09.023 },
    year = {2011},
    month = Jan,
    pdf = {http://hal.archives-ouvertes.fr/hal-00633629/PDF/Aseervatham-PatternRecognitionLetter2011.pdf},
}

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="article"><head>Articles dans des revues avec comité de lecture</head>
<biblStruct type="article" xml:id="hal-00633629"><analytic>
<author><persName><forename>Sujeevan</forename><surname>Aseervatham</surname></persName>
<affiliation>
<orgName type="laboratory">LIPN</orgName>
<orgName type="team">Laboratoire d'informatique de Paris-nord</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR7030</orgName>
<orgName type="institution">Université Paris XIII - Paris Nord</orgName>
</affiliation>
</author>
<author><persName><forename>Anestis</forename><surname>Antoniadis</surname></persName>
<affiliation>
<orgName type="laboratory">LJK</orgName>
<orgName type="team">Laboratoire Jean Kuntzmann</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5224</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Université Pierre-Mendès-France - Grenoble II</orgName>
<orgName type="institution">Institut Polytechnique de Grenoble - Grenoble Institute of Technology</orgName>
</affiliation>
</author>
<author><persName><forename>Éric</forename><surname>Gaussier</surname></persName>
<affiliation>
<orgName type="laboratory">LIG</orgName>
<orgName type="team">Laboratoire d'Informatique de Grenoble</orgName>
<country>FR</country>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut Polytechnique de Grenoble - Grenoble Institute of Technology</orgName>
<orgName type="institution">Université Pierre-Mendès-France - Grenoble II</orgName>
<orgName type="institution">CNRS : UMR5217</orgName>
</affiliation>
</author>
<author><persName><forename>Michel</forename><surname>Burlet</surname></persName>
<affiliation>
<orgName type="laboratory">Leibniz - IMAG</orgName>
<orgName type="team">Laboratoire Leibniz</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5522</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut National Polytechnique de Grenoble (INPG)</orgName>
</affiliation>
<affiliation>
<orgName type="laboratory">G-SCOP</orgName>
<orgName type="team">Laboratoire des sciences pour la conception, l'optimisation et la production</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5272</orgName>
<orgName type="institution">Institut National Polytechnique de Grenoble (INPG)</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
</affiliation>
</author>
<author><persName><forename>Yves</forename><surname>Denneulin</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Grenoble Rhône-Alpes / LIG laboratoire d'Informatique de Grenoble</orgName>
<orgName type="team">MESCAL</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5217</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Université Pierre-Mendès-France - Grenoble II</orgName>
<orgName type="institution">Institut polytechnique de Grenoble (Grenoble INP)</orgName>
</affiliation>
</author>
<title type="main" level="a">A sparse version of the ridge logistic regression for large-scale text categorization</title></analytic><monogr><title level="j" type="main">Pattern Recognition Letters</title><imprint><biblScope type="publicationDate">2011-01-15</biblScope><biblScope type="vol">32</biblScope><biblScope type="issue">2</biblScope><biblScope type="pp">101-106</biblScope></imprint></monogr><idno type="doi">10.1016/j.patrec.2010.09.023</idno><idno type="HALid">http://hal.archives-ouvertes.fr/hal-00633629</idno><idno type="HALFile">http://hal.archives-ouvertes.fr/hal-00633629/PDF/Aseervatham-PatternRecognitionLetter2011.pdf</idno></biblStruct>
</listBibl>
</listBibl>