Towards a True Acoustic-Visual Speech Synthesis

Asterios Toutios 1 Utpala Musti 1 Slim Ouni 1, * Vincent Colotte 1 Brigitte Wrobel-Dautcourt 2 Marie-Odile Berger 2
* Corresponding author
1 PAROLE - Analysis, perception and recognition of speech
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
2 MAGRIT - Visual Augmentation of Complex Environments
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : This paper presents an initial bimodal acoustic-visual synthesis system able to generate concurrently the speech signal and a 3D animation of the speaker's face. This is done by concatenating bimodal diphone units that consist of both acoustic and visual information. The latter is acquired using a stereovision technique. The proposed method addresses the problems of asyn- chrony and incoherence inherent in classic approaches to audiovisual synthesis. Unit selection is based on classic target and join costs from acoustic-only synthesis, which are augmented with a visual join cost. Preliminary results indicate the benefits of this approach, since both the synthesized speech signal and the face animation are of good quality.
Keywords : audiovisual speech synthesis talking head bimodal unit concatenation diphones
Document type :
Conference papers
Complete list of metadatas

Cited literature [16 references]  Display  Hide  Download
https://hal.inria.fr/inria-00526782
Contributor : Slim Ouni <>
Submitted on : Friday, October 15, 2010 - 5:08:01 PM
Last modification on : Thursday, January 11, 2018 - 6:20:14 AM
Long-term archiving on: Monday, January 17, 2011 - 10:55:05 AM

File

AVSP10-AT.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00526782, version 1

Collections

CNRS | INRIA | LORIA | UNIV-LORRAINE

Citation

Asterios Toutios, Utpala Musti, Slim Ouni, Vincent Colotte, Brigitte Wrobel-Dautcourt, et al.. Towards a True Acoustic-Visual Speech Synthesis. 9th International Conference on Auditory-Visual Speech Processing - AVSP2010, Sep 2010, Hakone, Kanagawa, Japan. pp.POS1-8. ⟨inria-00526782⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

575

Files downloads

222

 

 

Contact

archive-ouverte@inria.fr