Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication - Archive ouverte HAL Access content directly
Journal Articles Speech Communication Year : 2018

Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication

(1, 2) , (3)
1
2
3

Abstract

In audiovisual speech communication, the lower part of the face (mainly lips and jaw) actively participates during speech production. Modeling well lip motion and deformation in audiovisual speech synthesis is important to achieve realism and effective communication. This is essential for challenged population as hard-of-hearing people or new language learners. In this scope, we propose a technique that allows for animation of a human face with realistic lips using a limited number of control points. We have used an articu-lograph that provides high temporal and spatial precision, allowing tracking the positions of small electromagnetic sensors, even when occluded, which is often the case when tracking the lip movement. In our work, the control point data are first acquired, then fitted to a 3D face model of a human speaker, i.e., each control point is associated with a region of the face by minimizing the distance between the control points and the surface of the face model. Finally, we apply an interpolation scheme of the displacement field between the control points. This displacement field describes the deformation of a surface. In the case of the face, this method is well adapted to animating the region of the face that is highly correlated with speech, specifically the lips and the lower part of the face, even with a very limited number of control points.
Fichier principal
Vignette du fichier
lipControl-draft-specom-2015-rev1.compressed.pdf (393.52 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01631397 , version 1 (09-11-2017)

Identifiers

Cite

Slim Ouni, Guillaume Gris. Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication. Speech Communication, 2018, 96, pp.49-57. ⟨10.1016/j.specom.2017.11.006⟩. ⟨hal-01631397⟩
421 View
418 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More