Skip to Main content Skip to Navigation
New interface
Conference papers

Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated

Vinicius Ribeiro 1 Karyna Isaieva 2 Justine Leclère 2 Pierre-André Vuissoz 2 Yves Laprie 1 
1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : In this work, we address the prediction of speech articulators' temporal geometric position from the sequence of phonemes to be articulated. We start from a set of real-time MRI sequences uttered by a female French speaker. The contours of five articulators were tracked automatically in each of the frames in the MRI video. Then, we explore the capacity of a bidirectional GRU to correctly predict each articulator's shape and position given the sequence of phonemes and their duration. We propose a 5-fold cross-validation experiment to evaluate the generalization capacity of the model. In a second experiment, we evaluate our model's data efficiency by reducing training data. We evaluate the point-to-point Euclidean distance and the Pearson's correlations along time between the predicted and the target shapes. We also evaluate produced shapes of the critical articulators of specific phonemes. We show that our model can achieve good results with minimal data, producing very realistic vocal tract shapes.
Document type :
Conference papers
Complete list of metadata
Contributor : Vinicius de Paulo Souza Ribeiro Connect in order to contact the contributor
Submitted on : Thursday, September 30, 2021 - 3:54:42 PM
Last modification on : Friday, August 5, 2022 - 9:26:17 AM
Long-term archiving on: : Friday, December 31, 2021 - 8:50:13 PM


Files produced by the author(s)



Vinicius Ribeiro, Karyna Isaieva, Justine Leclère, Pierre-André Vuissoz, Yves Laprie. Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated. INTERSPEECH 2021, Aug 2021, Brno, Czech Republic. ⟨10.21437/Interspeech.2021-184⟩. ⟨hal-03360113⟩



Record views


Files downloads