Conditional Variational Auto-Encoder for Text-Driven Expressive AudioVisual Speech Synthesis - Archive ouverte HAL Access content directly
Conference Papers Year :

Conditional Variational Auto-Encoder for Text-Driven Expressive AudioVisual Speech Synthesis

(1) , (1) , (1) , (1)
1

Abstract

In recent years, the performance of speech synthesis systems has been improved thanks to deep learning-based models, but generating expressive audiovisual speech is still an open issue. The variational auto-encoders (VAE)s are recently proposed to learn latent representations of data. In this paper, we present a system for expressive text-to-audiovisual speech synthesis that learns a latent embedding space of emotions using a conditional generative model based on the variational auto-encoder framework. When conditioned on textual input, the VAE is able to learn an embedded representation that captures emotion characteristics from the signal, while being invariant to the phonetic content of the utterances. We applied this method in an unsuper-vised manner to generate duration, acoustic and visual features of speech. This conditional variational auto-encoder (CVAE) has been used to blend emotions together. This model was able to generate nuances of a given emotion or to generate new emotions that do not exist in our database. We conducted three perceptive experiments to evaluate our findings.
Fichier principal
Vignette du fichier
interspeech19_sd_26_06.pdf (1.34 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02175776 , version 1 (06-07-2019)

Identifiers

  • HAL Id : hal-02175776 , version 1

Cite

Sara Dahmani, Vincent Colotte, Valérian Girard, Slim Ouni. Conditional Variational Auto-Encoder for Text-Driven Expressive AudioVisual Speech Synthesis. INTERSPEECH 2019 - 20th Annual Conference of the International Speech Communication Association, Sep 2019, Graz, Austria. ⟨hal-02175776⟩
567 View
792 Download

Share

Gmail Facebook Twitter LinkedIn More