Animation of generic 3D Head models driven by speech

Lucas Terissi; Mauricio Cerda; Juan C. Gomez; Nancy Hitschfeld-Kahler; Bernard Girau; Renato Valenzuela

hal-00587016, version 1

Animation of generic 3D Head models driven by speech

Lucas Terissi () ¹, Mauricio Cerda ()^a^, 2, Juan C. Gomez ()^b^, 1, Nancy Hitschfeld-Kahler ()^c^, 3, Bernard Girau () ², Renato Valenzuela ()^c^, 3

IEEE International Conference on Multimedia and Expo - ICME 2011 (2011) To appear

Abstract: In this paper, a system for speech-driven animation of generic 3D head models is presented. The system is based on the inversion of a joint Audio-Visual Hidden Markov Model to estimate the visual information from speech data. Estimated visual speech features are used to animate a simple face model. The animation of a more complex head model is then obtained by automatically mapping the deformation of the simple model to it. The proposed algorithm allows the animation of 3D head models of arbitrary complexity through a simple setup procedure. The resulting animation is evaluated in terms of intelligibility of visual speech through subjective tests, showing a promising performance.

a – INRIA
b – Universidad Nacional de Rosario
c – Universidad de Chile
1: Laboratory for System Dynamics and Signal Processing
Universidad Nacional de Rosario – CIFASIS - CONICET
2: CORTEX (INRIA Lorraine - LORIA)
INRIA – CNRS : UMR7503 – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)
3: Departemento de Ciencias de la Computacion (DCC)
Universidad de Chile

Collaboration : stic amsud BAVI (09STIC06)

hal-00587016, version 1
http://hal.archives-ouvertes.fr/hal-00587016
oai:hal.archives-ouvertes.fr:hal-00587016
From: Mauricio Cerda <>
Submitted on: Tuesday, 19 April 2011 10:01:36
Updated on: Monday, 16 May 2011 14:48:39

See detailed view

Associated documents

PDF :

Export

Bibtex EndNote TEI RefWorks

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="inproceedings"><head>Peer-reviewed conferences/proceedings</head>
<biblStruct type="inproceedings" xml:id="hal-00587016"><analytic>
<author><persName><forename>Lucas</forename><surname>Terissi</surname></persName>
<affiliation>
<orgName type="team">Laboratory for System Dynamics and Signal Processing</orgName>
<country>AR</country>
<orgName type="institution">Universidad Nacional de Rosario</orgName>
<orgName type="institution">CIFASIS - CONICET</orgName>
</affiliation>
</author>
<author><persName><forename>Mauricio</forename><surname>Cerda</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">CORTEX</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Juan C.</forename><surname>Gomez</surname></persName>
<affiliation>
<orgName type="team">Laboratory for System Dynamics and Signal Processing</orgName>
<country>AR</country>
<orgName type="institution">Universidad Nacional de Rosario</orgName>
<orgName type="institution">CIFASIS - CONICET</orgName>
</affiliation>
</author>
<author><persName><forename>Nancy</forename><surname>Hitschfeld-Kahler</surname></persName>
<affiliation>
<orgName type="laboratory">DCC</orgName>
<orgName type="team">Departemento de Ciencias de la Computacion</orgName>
<country>CL</country>
<orgName type="institution">Universidad de Chile</orgName>
</affiliation>
</author>
<author><persName><forename>Bernard</forename><surname>Girau</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Lorraine - LORIA</orgName>
<orgName type="team">CORTEX</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR7503</orgName>
<orgName type="institution">Université Henri Poincaré - Nancy I</orgName>
<orgName type="institution">Université Nancy II</orgName>
<orgName type="institution">Institut National Polytechnique de Lorraine (INPL)</orgName>
</affiliation>
</author>
<author><persName><forename>Renato</forename><surname>Valenzuela</surname></persName>
<affiliation>
<orgName type="laboratory">DCC</orgName>
<orgName type="team">Departemento de Ciencias de la Computacion</orgName>
<country>CL</country>
<orgName type="institution">Universidad de Chile</orgName>
</affiliation>
</author>
<title type="main" level="a">Animation of generic 3D Head models driven by speech</title></analytic><monogr><title level="m" type="main">ICME 2011</title><meeting>IEEE International Conference on Multimedia and Expo - ICME 2011</meeting><imprint><publisher>IEEE</publisher><pubPlace><settlement>Barcelone</settlement><country>Spain</country></pubPlace><biblScope type="publicationDate">2011-07-15</biblScope><biblScope type="pp">To appear</biblScope></imprint></monogr><idno type="HALid">http://hal.archives-ouvertes.fr/hal-00587016</idno><idno type="HALFile">http://hal.archives-ouvertes.fr/hal-00587016/PDF/ICME_2011_DRAFT.pdf</idno></biblStruct>
</listBibl>
</listBibl>