Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication

Abstract : In audiovisual speech communication, the lower part of the face (mainly lips and jaw) actively participates during speech production. Modeling well lip motion and deformation in audiovisual speech synthesis is important to achieve realism and effective communication. This is essential for challenged population as hard-of-hearing people or new language learners. In this scope, we propose a technique that allows for animation of a human face with realistic lips using a limited number of control points. We have used an articu-lograph that provides high temporal and spatial precision, allowing tracking the positions of small electromagnetic sensors, even when occluded, which is often the case when tracking the lip movement. In our work, the control point data are first acquired, then fitted to a 3D face model of a human speaker, i.e., each control point is associated with a region of the face by minimizing the distance between the control points and the surface of the face model. Finally, we apply an interpolation scheme of the displacement field between the control points. This displacement field describes the deformation of a surface. In the case of the face, this method is well adapted to animating the region of the face that is highly correlated with speech, specifically the lips and the lower part of the face, even with a very limited number of control points.
Liste complète des métadonnées

Littérature citée [38 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01631397
Contributeur : Slim Ouni <>
Soumis le : jeudi 9 novembre 2017 - 10:51:38
Dernière modification le : mercredi 11 juillet 2018 - 01:26:59
Document(s) archivé(s) le : samedi 10 février 2018 - 12:24:04

Fichier

lipControl-draft-specom-2015-r...
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Slim Ouni, Guillaume Gris. Dynamic Lip Animation from a Limited number of Control Points: Towards an Effective Audiovisual Spoken Communication. Speech Communication, Elsevier : North-Holland, 2018, 96, pp.49-57. 〈10.1016/j.specom.2017.11.006〉. 〈hal-01631397〉

Partager

Métriques

Consultations de la notice

403

Téléchargements de fichiers

180