Transfer learning of the expressivity using flow metric learning in multispeaker text-to-speech synthesis - Archive ouverte HAL Access content directly
Conference Papers Year :

Transfer learning of the expressivity using flow metric learning in multispeaker text-to-speech synthesis

(1) , (1) , (1)
1

Abstract

In this paper, we present a novel deep metric learning architecture along with variational inference incorporated in a paramet-ric multispeaker expressive text-to-speech (TTS) system. We proposed inverse autoregressive flow (IAF) as a way to perform the variational inference, thus providing flexible approximate posterior distribution. The proposed approach condition the text-to-speech system on speaker embeddings so that latent space represents the emotion as semantic characteristics. For representing the speaker, we extracted speaker em-beddings from the x-vector based speaker recognition model trained on speech data from many speakers. To predict the vocoder features, we used the acoustic model conditioned on the textual features as well as on the speaker embedding. We transferred the expressivity by using the mean of the latent variables for each emotion to generate expressive speech in different speaker's voices for which no expressive speech data is available. We compared the results obtained using flow-based variational inference with variational autoencoder as a base-line model. The performance measured by mean opinion score (MOS), speaker MOS, and expressive MOS shows that N-pair loss based deep metric learning along with IAF model improves the transfer of expressivity in the desired speaker's voice in synthesized speech.
Fichier principal
Vignette du fichier
interspeech_2020_published_version.pdf (333.9 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02572106 , version 1 (13-05-2020)
hal-02572106 , version 2 (15-05-2020)
hal-02572106 , version 3 (22-10-2020)

Identifiers

  • HAL Id : hal-02572106 , version 3

Cite

Ajinkya Kulkarni, Vincent Colotte, Denis Jouvet. Transfer learning of the expressivity using flow metric learning in multispeaker text-to-speech synthesis. INTERSPEECH 2020, Oct 2020, Shanghai / Virtual, China. ⟨hal-02572106v3⟩
291 View
539 Download

Share

Gmail Facebook Twitter LinkedIn More