Skip to Main content Skip to Navigation
Journal articles

Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication

Abstract : In audiovisual speech communication, the lower part of the face (mainly lips and jaw) actively participates during speech production. Modeling well lip motion and deformation in audiovisual speech synthesis is important to achieve realism and effective communication. This is essential for challenged population as hard-of-hearing people or new language learners. In this scope, we propose a technique that allows for animation of a human face with realistic lips using a limited number of control points. We have used an articu-lograph that provides high temporal and spatial precision, allowing tracking the positions of small electromagnetic sensors, even when occluded, which is often the case when tracking the lip movement. In our work, the control point data are first acquired, then fitted to a 3D face model of a human speaker, i.e., each control point is associated with a region of the face by minimizing the distance between the control points and the surface of the face model. Finally, we apply an interpolation scheme of the displacement field between the control points. This displacement field describes the deformation of a surface. In the case of the face, this method is well adapted to animating the region of the face that is highly correlated with speech, specifically the lips and the lower part of the face, even with a very limited number of control points.
Complete list of metadata

Cited literature [38 references]  Display  Hide  Download
Contributor : Slim Ouni Connect in order to contact the contributor
Submitted on : Thursday, November 9, 2017 - 10:51:38 AM
Last modification on : Thursday, January 20, 2022 - 5:26:12 PM
Long-term archiving on: : Saturday, February 10, 2018 - 12:24:04 PM


Files produced by the author(s)




Slim Ouni, Guillaume Gris. Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication. Speech Communication, Elsevier : North-Holland, 2018, 96, pp.49-57. ⟨10.1016/j.specom.2017.11.006⟩. ⟨hal-01631397⟩



Les métriques sont temporairement indisponibles