Skip to Main content Skip to Navigation
Journal articles

Expression Recognition with Deep Features Extracted from Holistic and Part-based Models

S L Happy 1 Antitza Dantcheva 1 Francois Bremond 1
1 STARS - Spatio-Temporal Activity Recognition Systems
CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : Facial expression recognition aims to accurately interpret facial muscle movements in affective states (emotions). Previous studies have proposed holistic analysis of the face, as well as the extraction of features pertained only to specific facial regions towards expression recognition. While classically the latter have shown better performances, we here explore this in the context of deep learning. In particular, this work provides a performance comparison of holistic and part-based deep learning models for expression recognition. In addition, we showcase the effectiveness of skip connections, which allow a network to infer from both low and high-level feature maps. Our results suggest that holistic models outperform part-based models, in the absence of skip connections. Finally, based on our findings, we propose a data augmentation scheme, which we incorporate in a part-based model. The proposed multi-face multi-part (MFMP) model leverages the wide information from part-based data augmentation, where we train the network using the facial parts extracted from different face samples of the same expression class. Extensive experiments on publicly available datasets show a significant improvement of facial expression classification with the proposed MFMP framework.
Document type :
Journal articles
Complete list of metadata

Cited literature [73 references]  Display  Hide  Download
Contributor : Antitza Dantcheva Connect in order to contact the contributor
Submitted on : Tuesday, October 20, 2020 - 11:01:46 AM
Last modification on : Thursday, January 21, 2021 - 2:32:02 PM
Long-term archiving on: : Thursday, January 21, 2021 - 6:47:32 PM


Files produced by the author(s)


  • HAL Id : hal-02972172, version 1


S L Happy, Antitza Dantcheva, Francois Bremond. Expression Recognition with Deep Features Extracted from Holistic and Part-based Models. Image and Vision Computing, Elsevier, 2020. ⟨hal-02972172⟩



Record views


Files downloads