Structured Penalties for Log-linear Language Models - Archive ouverte HAL Access content directly
Conference Papers Year : 2013

Structured Penalties for Log-linear Language Models

(1) , (1) , (2) , (3, 4) , (1)
1
2
3
4

Abstract

Language models can be formalized as loglinear regression models where the input features represent previously observed contexts up to a certain length m. The complexity of existing algorithms to learn the parameters by maximum likelihood scale linearly in nd, where n is the length of the training corpus and d is the number of observed features. We present a model that grows logarithmically in d, making it possible to efficiently leverage longer contexts. We account for the sequential structure of natural language using treestructured penalized objectives to avoid overfitting and achieve better generalization.
Fichier principal
Vignette du fichier
anil_emnlp.pdf (195.54 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-00904820 , version 1 (15-11-2013)

Identifiers

  • HAL Id : hal-00904820 , version 1

Cite

Anil Nelakanti, Cédric Archambeau, Julien Mairal, Francis Bach, Guillaume Bouchard. Structured Penalties for Log-linear Language Models. EMNLP - Empirical Methods in Natural Language Processing, Oct 2013, Seattle, United States. pp.233-243. ⟨hal-00904820⟩
471 View
174 Download

Share

Gmail Facebook Twitter LinkedIn More