Skip to Main content Skip to Navigation
Conference papers

Towards a True Acoustic-Visual Speech Synthesis

Asterios Toutios 1 Utpala Musti 1 Slim Ouni 1, * Vincent Colotte 1 Brigitte Wrobel-Dautcourt 2 Marie-Odile Berger 2
* Corresponding author
1 PAROLE - Analysis, perception and recognition of speech
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
2 MAGRIT - Visual Augmentation of Complex Environments
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : This paper presents an initial bimodal acoustic-visual synthesis system able to generate concurrently the speech signal and a 3D animation of the speaker's face. This is done by concatenating bimodal diphone units that consist of both acoustic and visual information. The latter is acquired using a stereovision technique. The proposed method addresses the problems of asyn- chrony and incoherence inherent in classic approaches to audiovisual synthesis. Unit selection is based on classic target and join costs from acoustic-only synthesis, which are augmented with a visual join cost. Preliminary results indicate the benefits of this approach, since both the synthesized speech signal and the face animation are of good quality.
Complete list of metadata

Cited literature [16 references]  Display  Hide  Download
Contributor : Slim Ouni Connect in order to contact the contributor
Submitted on : Friday, October 15, 2010 - 5:08:01 PM
Last modification on : Thursday, January 20, 2022 - 5:30:25 PM
Long-term archiving on: : Monday, January 17, 2011 - 10:55:05 AM


Files produced by the author(s)


  • HAL Id : inria-00526782, version 1



Asterios Toutios, Utpala Musti, Slim Ouni, Vincent Colotte, Brigitte Wrobel-Dautcourt, et al.. Towards a True Acoustic-Visual Speech Synthesis. 9th International Conference on Auditory-Visual Speech Processing - AVSP2010, Sep 2010, Hakone, Kanagawa, Japan. pp.POS1-8. ⟨inria-00526782⟩



Les métriques sont temporairement indisponibles