Visual Coding and Tracking of Speech Related Facial Motion

Abstract : We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip raising, to represent the primary visual gestures in speech articulation. To generate a parametric model of facial actions, first a statistical model is developed by analyzing accurate 3D data of a reference human subject. The FSP are then associated to the linear modes of this statistical model resulting in a 3D parametric facial mesh that is linearly deformed using FSP. For tracking of talking facial motions, the parametric model is adapted and aligned to a subject's face. Then the face motion is tracked by optimally aligning the incoming video frames with the face model, textured with the first image, and deformed by varying the FSP, head rotations, and translations. Finer details of lip and skin deformation are modeled using a blend of textures into an appearance model. We show results of the tracking for different subjects using our method. Finally, we demonstrate the facial activity encoding into the four FSP values to represent speaker-independent phonetic information and to generate different styles of animation.
Type de document :
Communication dans un congrès
IEEE CVPR International Workshop on Cues in Communication, Dec 2001, Honolulu, United States. 2001
Liste complète des métadonnées

Littérature citée [32 références]  Voir  Masquer  Télécharger


https://hal.inria.fr/inria-00389357
Contributeur : Lionel Reveret <>
Soumis le : jeudi 28 mai 2009 - 15:56:25
Dernière modification le : mercredi 3 novembre 2010 - 15:07:59
Document(s) archivé(s) le : jeudi 10 juin 2010 - 23:59:14

Fichiers

reveret-essa-speech.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00389357, version 1

Citation

Lionel Reveret, Irfan Essa. Visual Coding and Tracking of Speech Related Facial Motion. IEEE CVPR International Workshop on Cues in Communication, Dec 2001, Honolulu, United States. 2001. 〈inria-00389357〉

Partager

Métriques

Consultations de la notice

80

Téléchargements de fichiers

116