From FreEM to D'AlemBERT - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

From FreEM to D'AlemBERT

Résumé

Language models for historical states of language are becoming increasingly important to allow the optimal digitisation and analysis of old textual sources. Because these historical states are at the same time more complex to process and more scarce in the corpora available, specific efforts are necessary to train natural language processing (NLP) tools adapted to the data. In this paper, we present our efforts to develop NLP tools for Early Modern French (historical French from the 16th to the 18th centuries). We present the FreEMmax corpus of Early Modern French and D'AlemBERT, a RoBERTa-based language model trained on FreEMmax. We evaluate the usefulness of D'AlemBERT by fine-tuning it on a part-of-speech tagging task, outperforming previous work on the test set. Importantly, we find evidence for the transfer learning capacity of the language model, since its performance on lesser-resourced time periods appears to have been boosted by the more resourced ones. We release D'AlemBERT and the open-sourced subpart of the FreEMmax corpus.
Fichier principal
Vignette du fichier
D_AlemBERT.pdf (329.57 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03596653 , version 1 (14-10-2022)

Licence

Paternité

Identifiants

Citer

Simon Gabay, Pedro Ortiz Suarez, Alexandre Bartz, Alix Chagué, Rachel Bawden, et al.. From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French. 13th Language Resources and Evaluation Conference - LREC 2022, European Language Resources Association, Jun 2022, Marseille, France. pp.3367-3374. ⟨hal-03596653⟩
260 Consultations
84 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More