The Impact of Specialized Corpora for Word Embeddings in Natural Langage Understanding. - Archive ouverte HAL Access content directly
Journal Articles Studies in Health Technology and Informatics Year : 2020

The Impact of Specialized Corpora for Word Embeddings in Natural Langage Understanding.

(1, 2, 3) , (2, 4) , (5) , (3) , (6) , (3)
1
2
3
4
5
6

Abstract

Recent studies in the biomedical domain suggest that learning statistical word representations (static or contextualized word embeddings) on large corpora of specialized data improve the results on downstream natural language processing (NLP) tasks. In this paper, we explore the impact of the data source of word representations on a natural language understanding task. We compared embeddings learned with Fasttext (static embedding) and ELMo (contextualized embedding) representations, learned either on the general domain (Wikipedia) or on specialized data (electronic health records, EHR). The best results were obtained with ELMo representations learned on EHR data for the two sub-tasks (+7% and +4% of gain in F1-score). Moreover, ELMo representations were trained with only a fraction of the data used for Fasttext.
Not file

Dates and versions

hal-03476839 , version 1 (13-12-2021)

Identifiers

Cite

Antoine Neuraz, Bastien Rance, Nicolas Garcelon, Leonardo Campillos Llanos, Anita Burgun, et al.. The Impact of Specialized Corpora for Word Embeddings in Natural Langage Understanding.. Studies in Health Technology and Informatics, 2020, 270, pp.432-436. ⟨10.3233/SHTI200197⟩. ⟨hal-03476839⟩
20 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More