Skip to Main content Skip to Navigation
Conference papers

Visual Coding and Tracking of Speech Related Facial Motion

Abstract : We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip raising, to represent the primary visual gestures in speech articulation. To generate a parametric model of facial actions, first a statistical model is developed by analyzing accurate 3D data of a reference human subject. The FSP are then associated to the linear modes of this statistical model resulting in a 3D parametric facial mesh that is linearly deformed using FSP. For tracking of talking facial motions, the parametric model is adapted and aligned to a subject's face. Then the face motion is tracked by optimally aligning the incoming video frames with the face model, textured with the first image, and deformed by varying the FSP, head rotations, and translations. Finer details of lip and skin deformation are modeled using a blend of textures into an appearance model. We show results of the tracking for different subjects using our method. Finally, we demonstrate the facial activity encoding into the four FSP values to represent speaker-independent phonetic information and to generate different styles of animation.
Document type :
Conference papers
Complete list of metadata

Cited literature [32 references]  Display  Hide  Download


https://hal.inria.fr/inria-00389357
Contributor : Lionel Reveret <>
Submitted on : Thursday, May 28, 2009 - 3:56:25 PM
Last modification on : Wednesday, November 3, 2010 - 3:07:59 PM
Long-term archiving on: : Thursday, June 10, 2010 - 11:59:14 PM

Files

reveret-essa-speech.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00389357, version 1

Citation

Lionel Reveret, Irfan Essa. Visual Coding and Tracking of Speech Related Facial Motion. IEEE CVPR International Workshop on Cues in Communication, Dec 2001, Honolulu, United States. ⟨inria-00389357⟩

Share

Metrics

Record views

105

Files downloads

175