An alternative scheme for perplexity estimation and its assessment for the evaluation of language models - Archive ouverte HAL Access content directly
Journal Articles Computer Speech and Language Year : 2001

An alternative scheme for perplexity estimation and its assessment for the evaluation of language models

(1) , (2) , (2) , (3) , (4) , (4)
1
2
3
4

Abstract

Language models are usually evaluated on test texts using the perplexity derived from the likelihood function computed on these texts (test set perplexity). In order to use this measure in the framework of a comparative evaluation campaign, we have developed an alternative scheme for estimating the test set perplexity. The method is derived from the Shannon game and based on a gambling approach on the next word to come in a truncated sentence. We also study the entropy bounds proposed by Shannon and based on the rank of the correct answer, in order to estimate a perplexity interval for non-probabilistic language models. The relevance of the approach is validated on an example. We then report the results of a preliminary comparative evaluation using the proposed scheme
Not file

Dates and versions

inria-00100687 , version 1 (26-09-2006)

Identifiers

  • HAL Id : inria-00100687 , version 1

Cite

Frédéric Bimbot, Marc El Bèze, Stéphane Igounet, Michèle Jardino, Kamel Smaïli, et al.. An alternative scheme for perplexity estimation and its assessment for the evaluation of language models. Computer Speech and Language, 2001, 15 (1), pp.1-13. ⟨inria-00100687⟩
321 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More