Online Learning of Task-specific Word Representations with a Joint Biconvex Passive-Aggressive Algorithm

Pascal Denis 1 Liva Ralaivola 2
1 MAGNET - Machine Learning in Information Networks
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
2 QARMA - éQuipe AppRentissage et MultimediA [Marseille]
LIF - Laboratoire d'informatique Fondamentale de Marseille
Abstract : This paper presents a new, efficient method for learning task-specific word vectors using a variant of the Passive-Aggressive algorithm. Specifically, this algorithm learns a word embedding matrix in tandem with the classifier parameters in an online fashion, solving a bi-convex constrained optimization at each iteration. We provide a theoretical analysis of this new algorithm in terms of regret bounds, and evaluate it on both synthetic data and NLP classification problems, including text classification and sentiment analysis. In the latter case, we compare various pre-trained word vectors to initialize our word embedding matrix, and show that the matrix learned by our algorithm vastly outperforms the initial matrix, with performance results comparable or above the state-of-the-art on these tasks.
Type de document :
Communication dans un congrès
European Chapter of the Association for Computational Linguistics, Apr 2017, Valencia, Spain. pp.775 - 784, 2017, 〈10.18653/v1/E17-1073〉
Liste complète des métadonnées

Littérature citée [23 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01590594
Contributeur : Pascal Denis <>
Soumis le : mardi 19 septembre 2017 - 17:06:28
Dernière modification le : mardi 3 juillet 2018 - 11:39:11

Fichier

eacl17_rpa.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Pascal Denis, Liva Ralaivola. Online Learning of Task-specific Word Representations with a Joint Biconvex Passive-Aggressive Algorithm. European Chapter of the Association for Computational Linguistics, Apr 2017, Valencia, Spain. pp.775 - 784, 2017, 〈10.18653/v1/E17-1073〉. 〈hal-01590594〉

Partager

Métriques

Consultations de la notice

320

Téléchargements de fichiers

78