Skip to Main content Skip to Navigation
Conference papers

Few-shot learning through contextual data augmentation

Abstract : Machine translation (MT) models used in industries with constantly changing topics, such as translation or news agencies, need to adapt to new data to maintain their performance over time. Our aim is to teach a pre-trained MT model to translate previously unseen words accurately, based on very few examples. We propose (i) an experimental setup allowing us to simulate novel vocabulary appearing in human-submitted translations, and (ii) corresponding evaluation metrics to compare our approaches. We extend a data augmentation approach using a pre-trained language model to create training examples with similar contexts for novel words. We compare different fine-tuning and data augmentation approaches and show that adaptation on the scale of one to five examples is possible. Combining data augmentation with randomly selected training sentences leads to the highest BLEU score and accuracy improvements. Impressively, with only 1 to 5 examples, our model reports better accuracy scores than a reference system trained with on average 313 parallel examples.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03121971
Contributor : Rachel Bawden Connect in order to contact the contributor
Submitted on : Tuesday, January 26, 2021 - 5:21:27 PM
Last modification on : Friday, January 21, 2022 - 3:17:37 AM
Long-term archiving on: : Tuesday, April 27, 2021 - 7:28:09 PM

File

Lifelong_learning_EACL_2021_su...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03121971, version 1

Collections

Citation

Farid Arthaud, Rachel Bawden, Alexandra Birch. Few-shot learning through contextual data augmentation. EACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Apr 2021, Kiev / Virtual, Ukraine. ⟨hal-03121971⟩

Share

Metrics

Les métriques sont temporairement indisponibles