Skip to Main content Skip to Navigation
Conference papers

Visual Coding and Tracking of Speech Related Facial Motion

Abstract : We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip raising, to represent the primary visual gestures in speech articulation. To generate a parametric model of facial actions, first a statistical model is developed by analyzing accurate 3D data of a reference human subject. The FSP are then associated to the linear modes of this statistical model resulting in a 3D parametric facial mesh that is linearly deformed using FSP. For tracking of talking facial motions, the parametric model is adapted and aligned to a subject's face. Then the face motion is tracked by optimally aligning the incoming video frames with the face model, textured with the first image, and deformed by varying the FSP, head rotations, and translations. Finer details of lip and skin deformation are modeled using a blend of textures into an appearance model. We show results of the tracking for different subjects using our method. Finally, we demonstrate the facial activity encoding into the four FSP values to represent speaker-independent phonetic information and to generate different styles of animation.
Document type :
Conference papers
Complete list of metadata

Cited literature [32 references]  Display  Hide  Download
Contributor : Lionel Reveret Connect in order to contact the contributor
Submitted on : Thursday, May 28, 2009 - 3:56:25 PM
Last modification on : Sunday, February 20, 2022 - 5:44:01 PM
Long-term archiving on: : Thursday, June 10, 2010 - 11:59:14 PM


Files produced by the author(s)


  • HAL Id : inria-00389357, version 1


Lionel Reveret, Irfan Essa. Visual Coding and Tracking of Speech Related Facial Motion. IEEE CVPR International Workshop on Cues in Communication, Dec 2001, Honolulu, United States. ⟨inria-00389357⟩



Record views


Files downloads