An alternative scheme for perplexity estimation and its assessment for the evaluation of language models

Abstract : Language models are usually evaluated on test texts using the perplexity derived from the likelihood function computed on these texts (test set perplexity). In order to use this measure in the framework of a comparative evaluation campaign, we have developed an alternative scheme for estimating the test set perplexity. The method is derived from the Shannon game and based on a gambling approach on the next word to come in a truncated sentence. We also study the entropy bounds proposed by Shannon and based on the rank of the correct answer, in order to estimate a perplexity interval for non-probabilistic language models. The relevance of the approach is validated on an example. We then report the results of a preliminary comparative evaluation using the proposed scheme
Type de document :
Article dans une revue
Computer Speech and Language, Elsevier, 2001, 15 (1), pp.1-13
Liste complète des métadonnées

https://hal.inria.fr/inria-00100687
Contributeur : Publications Loria <>
Soumis le : mardi 26 septembre 2006 - 14:49:19
Dernière modification le : jeudi 11 janvier 2018 - 06:25:46

Identifiants

  • HAL Id : inria-00100687, version 1

Citation

Frédéric Bimbot, Marc El Bèze, Stéphane Igounet, Michèle Jardino, Kamel Smaïli, et al.. An alternative scheme for perplexity estimation and its assessment for the evaluation of language models. Computer Speech and Language, Elsevier, 2001, 15 (1), pp.1-13. 〈inria-00100687〉

Partager

Métriques

Consultations de la notice

347