Random simulations of a datatable for efficiently mining reliable and non-redundant itemsets

Abstract : Our goal is twofold: 1) we want to mine the only statistically valid 2-itemsets out of a boolean datatable, 2) on this basis, we want to build the only higher-order non-redundant itemsets compared to their sub-itemsets. For the first task we have designed a randomization test (Tournebool) respectful of the structure of the data variables and independant from the specific distributions of the data. In our test set (959 texts and 8477 terms), this leads to a reduction from 126, 000 2-itemsets to 13, 000 significant ones, at the 99% confidence interval. For the second task, we have devised a hierarchical stepwise procedure (MIDOVA) for evaluating the residual amount of variation devoted to higher-order itemsets, yielding new possible positive or negative high-order relations. On our example, this leads to counts of 7,712 for 2-itemsets to 3 for 6-itemsets, and no higher-order ones, in a computationally efficient way.
Type de document :
Communication dans un congrès
12th International Conference on Applied Stochastic Models and Data Analysis - ASMDA 2007, May 2007, Chania, Crête, Greece. 2007
Liste complète des métadonnées

Littérature citée [14 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00186100
Contributeur : Martine Cadot <>
Soumis le : mercredi 7 novembre 2007 - 22:02:38
Dernière modification le : jeudi 15 février 2018 - 08:48:10
Document(s) archivé(s) le : lundi 24 septembre 2012 - 15:00:50

Fichier

Cadot_Lelu_ASMDA2007.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00186100, version 1

Citation

Martine Cadot, Pascal Cuxac, Alain Lelu. Random simulations of a datatable for efficiently mining reliable and non-redundant itemsets. 12th International Conference on Applied Stochastic Models and Data Analysis - ASMDA 2007, May 2007, Chania, Crête, Greece. 2007. 〈inria-00186100〉

Partager

Métriques

Consultations de la notice

317

Téléchargements de fichiers

217