Realistic Face Animation for Audiovisual Speech Applications: A Densification Approach Driven by Sparse Stereo Meshes

Marie-Odile Berger 1 Jonathan Ponroy 2 Brigitte Wrobel-Dautcourt 1
1 MAGRIT - Visual Augmentation of Complex Environments
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
2 TRIO - Real time and interoperability
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Being able to produce realistic facial animation is crucial for many speech applications in language learning technologies. Reaching realism needs to acquire and to animate dense 3D models of the face which are often acquired with 3D scanners. However, acquiring the dy- namics of the speech from 3D scans is difficult as the acquisition time generally allows only sustained sounds to be recorded. On the contrary, acquiring the speech dynamics on a sparse set of points is easy using a stereovision recording a talker with markers painted on his/her face. In this paper, we propose an approach to animate a very realistic dense talking head which makes use of a reduced set of 3D dense meshes ac- quired for sustained sounds as well as the speech dynamics learned on a talker painted with white markers. The contributions of the paper are twofold: We first propose an appropriate principal component anal- ysis (PCA) with missing data techniques in order to compute the basic modes of the speech dynamics despite possible unobservable points in the sparse meshes obtained by the stereovision system. We then propose a method for densifying the modes, that is a method for computing the dense modes for spatial animation from the sparse modes learned by the stereovision system. Examples prove the effectiveness of the approach and the high realism obtained with our method.
Keywords : face animation densification PCA with missing data
Type de document :
Communication dans un congrès
André Gagalowicz. Computer Vision/Computer Graphics CollaborationTechniques 4th International Conference, MIRAGE 2009, May 2009, Rocquencourt, France. Springer Berlin / Heidelberg, 5496, pp.297-307, 2009, LNCS. 〈10.1007/978-3-642-01811-4_27〉
Liste complète des métadonnées

Littérature citée [11 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/inria-00429338
Contributeur : Marie-Odile Berger <>
Soumis le : mardi 11 mai 2010 - 17:54:06
Dernière modification le : jeudi 11 janvier 2018 - 06:20:14
Document(s) archivé(s) le : jeudi 16 septembre 2010 - 12:40:01

Fichier

mirage2009Final.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

INRIA | LORIA | UNIV-LORRAINE

Citation

Marie-Odile Berger, Jonathan Ponroy, Brigitte Wrobel-Dautcourt. Realistic Face Animation for Audiovisual Speech Applications: A Densification Approach Driven by Sparse Stereo Meshes. André Gagalowicz. Computer Vision/Computer Graphics CollaborationTechniques 4th International Conference, MIRAGE 2009, May 2009, Rocquencourt, France. Springer Berlin / Heidelberg, 5496, pp.297-307, 2009, LNCS. 〈10.1007/978-3-642-01811-4_27〉. 〈inria-00429338〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

304

Téléchargements de fichiers

263

Contact


archive-ouverte@inria.fr