Setup for Acoustic-Visual Speech Synthesis by Concatenating Bimodal Units

Asterios Toutios 1 Utpala Musti 1 Slim Ouni 1 Vincent Colotte 1 Brigitte Wrobel-Dautcourt 2 Marie-Odile Berger 2
1 PAROLE - Analysis, perception and recognition of speech
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
2 MAGRIT - Visual Augmentation of Complex Environments
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : This paper presents preliminary work on building a system able to synthesize concurrently the speech signal and a 3D animation of the speaker's face. This is done by concatenating bimodal diphone units, that is, units that comprise both acoustic and visual information. The latter is acquired using a stereovision technique. The proposed method addresses the problems of asyn- chrony and incoherence inherent in classic approaches to au- diovisual synthesis. Unit selection is based on classic target and join costs from acoustic-only synthesis, which are augmented with a visual join cost. Preliminary results indicate the benefits of the approach, since both the synthesized speech signal and the face animation are of good quality. Planned improvements and enhancements to the system are outlined.
Liste complète des métadonnées

Littérature citée [10 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00526766
Contributeur : Slim Ouni <>
Soumis le : vendredi 15 octobre 2010 - 16:45:41
Dernière modification le : jeudi 11 janvier 2018 - 06:20:14
Document(s) archivé(s) le : lundi 17 janvier 2011 - 10:52:11

Fichier

IS10-AT.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00526766, version 1

Collections

Citation

Asterios Toutios, Utpala Musti, Slim Ouni, Vincent Colotte, Brigitte Wrobel-Dautcourt, et al.. Setup for Acoustic-Visual Speech Synthesis by Concatenating Bimodal Units. Interspeech 2010, Sep 2010, Makuhari, Chiba, Japan. pp.486-489, 2010. 〈inria-00526766〉

Partager

Métriques

Consultations de la notice

295

Téléchargements de fichiers

178