Towards a True Acoustic-Visual Speech Synthesis

Asterios Toutios; Utpala Musti; Slim Ouni; Vincent Colotte; Brigitte Wrobel-Dautcourt; Marie-Odile Berger

inria-00526782, version 1

Towards a True Acoustic-Visual Speech Synthesis

Asterios Toutios ()^a^, 1, Utpala Musti ()^b^, 1, Slim Ouni ( Auteur à contacter de préférence ) ¹, Vincent Colotte ()^c^, 1, Brigitte Wrobel-Dautcourt ()^c^, 2, Marie-Odile Berger ()^b^, 2

9th International Conference on Auditory-Visual Speech Processing - AVSP2010 (2010) POS1-8

Résumé : This paper presents an initial bimodal acoustic-visual synthesis system able to generate concurrently the speech signal and a 3D animation of the speaker's face. This is done by concatenating bimodal diphone units that consist of both acoustic and visual information. The latter is acquired using a stereovision technique. The proposed method addresses the problems of asyn- chrony and incoherence inherent in classic approaches to audiovisual synthesis. Unit selection is based on classic target and join costs from acoustic-only synthesis, which are augmented with a visual join cost. Preliminary results indicate the benefits of this approach, since both the synthesized speech signal and the face animation are of good quality.

a – Université Nancy II
b – INRIA
c – Université Henri Poincaré - Nancy I
1 : PAROLE (INRIA Lorraine - LORIA)
INRIA – CNRS : UMR7503 – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)
2 : MAGRIT (INRIA Lorraine - LORIA)
CNRS : UMR7503 – INRIA – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)

inria-00526782, version 1
http://hal.inria.fr/inria-00526782
oai:hal.inria.fr:inria-00526782
Contributeur : Slim Ouni <>
Soumis le : Vendredi 15 Octobre 2010, 17:08:01
Dernière modification le : Lundi 25 Octobre 2010, 09:25:22

Voir la fiche détaillée

Documents associés

PDF :

Exporter

Bibtex EndNote TEI RefWorks

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="inproceedings"><head>Communications avec actes</head>
<biblStruct type="inproceedings" xml:id="inria-00526782"><analytic>
<author><persName><forename>Asterios</forename><surname>Toutios</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">PAROLE</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Utpala</forename><surname>Musti</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">PAROLE</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Slim</forename><surname>Ouni</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">PAROLE</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Vincent</forename><surname>Colotte</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">PAROLE</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Brigitte</forename><surname>Wrobel-Dautcourt</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">MAGRIT</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Marie-Odile</forename><surname>Berger</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">MAGRIT</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<title type="main" level="a">Towards a True Acoustic-Visual Speech Synthesis</title></analytic><monogr><meeting>9th International Conference on Auditory-Visual Speech Processing - AVSP2010</meeting><imprint><pubPlace><settlement>Hakone, Kanagawa</settlement><country>Japon</country></pubPlace><biblScope type="publicationDate">2010-09-27</biblScope><biblScope type="pp">POS1-8</biblScope></imprint></monogr><idno type="HALid">http://hal.inria.fr/inria-00526782</idno><idno type="HALFile">http://hal.inria.fr/inria-00526782/PDF/AVSP10-AT.pdf</idno></biblStruct>
</listBibl>
</listBibl>