Online Learning of Task-specific Word Representations with a Joint Biconvex Passive-Aggressive Algorithm - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Online Learning of Task-specific Word Representations with a Joint Biconvex Passive-Aggressive Algorithm

Pascal Denis

Résumé

This paper presents a new, efficient method for learning task-specific word vectors using a variant of the Passive-Aggressive algorithm. Specifically, this algorithm learns a word embedding matrix in tandem with the classifier parameters in an online fashion, solving a bi-convex constrained optimization at each iteration. We provide a theoretical analysis of this new algorithm in terms of regret bounds, and evaluate it on both synthetic data and NLP classification problems, including text classification and sentiment analysis. In the latter case, we compare various pre-trained word vectors to initialize our word embedding matrix, and show that the matrix learned by our algorithm vastly outperforms the initial matrix, with performance results comparable or above the state-of-the-art on these tasks.
Fichier principal
Vignette du fichier
eacl17_rpa.pdf (409.83 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01590594 , version 1 (19-09-2017)

Identifiants

Citer

Pascal Denis, Liva Ralaivola. Online Learning of Task-specific Word Representations with a Joint Biconvex Passive-Aggressive Algorithm. European Chapter of the Association for Computational Linguistics, Apr 2017, Valencia, Spain. pp.775 - 784, ⟨10.18653/v1/E17-1073⟩. ⟨hal-01590594⟩
305 Consultations
1169 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More