Skip to Main content Skip to Navigation
Conference papers

Cross-modal Deep Face Normals with Deactivable Skip Connections

Victoria Fernández Abrevaya 1 Adnane Boukhayma 2 Philip H. S. Torr 3 Edmond Boyer 1
1 MORPHEO - Capture and Analysis of Shapes in Motion
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
2 MIMETIC - Analysis-Synthesis Approach for Virtual Human Simulation
UR2 - Université de Rennes 2, Inria Rennes – Bretagne Atlantique , IRISA-D6 - MEDIA ET INTERACTIONS
Abstract : We present an approach for estimating surface normals from in-the-wild color images of faces. While datadriven strategies have been proposed for single face images, limited available ground truth data makes this problem difficult. To alleviate this issue, we propose a method that can leverage all available image and normal data, whether paired or not, thanks to a novel cross-modal learning architecture. In particular, we enable additional training with single modality data, either color or normal, by using two encoder-decoder networks with a shared latent space. The proposed architecture also enables face details to be transferred between the image and normal domains, given paired data, through skip connections between the image encoder and normal decoder. Core to our approach is a novel module that we call deactivable skip connections, which allows integrating both the auto-encoded and imageto-normal branches within the same architecture that can be trained end-to-end. This allows learning of a rich latent space that can accurately capture the normal information. We compare against state-of-the-art methods and show that our approach can achieve significant improvements, both quantitative and qualitative, with natural face images.
Complete list of metadatas

Cited literature [68 references]  Display  Hide  Download

https://hal.inria.fr/hal-02525049
Contributor : Victoria Fernandez Abrevaya <>
Submitted on : Monday, March 30, 2020 - 5:09:21 PM
Last modification on : Tuesday, November 24, 2020 - 4:00:09 PM

File

abrevaya-et-al__cvpr-20.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02525049, version 1

Citation

Victoria Fernández Abrevaya, Adnane Boukhayma, Philip H. S. Torr, Edmond Boyer. Cross-modal Deep Face Normals with Deactivable Skip Connections. CVPR 2020 - IEEE Conference on Computer Vision and Pattern Recognition, Jun 2020, Seattle, United States. pp.1-11. ⟨hal-02525049⟩

Share

Metrics

Record views

329

Files downloads

442