Skip to Main content Skip to Navigation
Conference papers

Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention

Abstract : Remote HR measurement HR Sleep monitor Health monitor Defection detection Stress analysis PRV … Fig. 1. Vision based remote HR measurement has various applications, such as sleep and health monitoring, defection detection, and stress analysis. While rPPG based HR measurement has shown promising results under controlled conditions [5], [6], [7], [8], less-constrained settings such as non-homogeneous illumination and the movement of the human pose remaining challenges in this context. At the same time, data driven methods, especially deep learning methods, have shown great mod-eling power and have achieved great success in many other applications such as object detection [9], image classification [10], as well as face recognition [11]. Several works have successfully leveraged the strong modeling ability of deep neural networks to the task of remote HR estimation [12], [13], [14]. However, all existing methods predominantly focused on learning a mapping function from the representation of face videos to the ground-truth HR, and failed to take the characteristics of rPPG based HR signal into consideration. As stated in Fig. 2, rPPG signals can produce biased result due to face movement and illumination lighting variations. This biasness will introduce great noise and greatly influence the learning producer. An automatic mechanism that can help to remove these polluted HR signals is required. To mitigate the aforementioned gap in this paper, we propose an end-to-end approach for remote HR measurement from faces utilizing a channel and spatial-temporal attention mechanism. We first utilize the spatial-temporal Abstract-In this work, we propose an end-to-end approach for robust remote heart rate (HR) measurement gleaned from facial videos. Specifically the approach is based on remote pho-toplethysmography (rPPG), which constitutes a pulse triggered perceivable chromatic variation, sensed in RGB-face videos. Consequently, rPPGs can be affected in less-constrained settings. To unpin the shortcoming, the proposed algorithm utilizes a spatio-temporal attention mechanism, which places focus on the salient features included in rPPG-signals. In addition, we propose an effective rPPG augmentation approach, generating multiple rPPG signals with varying HRs from a single face video. Experimental results on the public datasets VIPL-HR and MMSE-HR show that the proposed method outperforms state-of-the-art algorithms in remote HR estimation.
Complete list of metadata

Cited literature [31 references]  Display  Hide  Download
Contributor : Antitza Dantcheva <>
Submitted on : Tuesday, November 26, 2019 - 2:58:13 PM
Last modification on : Monday, December 14, 2020 - 5:32:20 PM


Files produced by the author(s)


  • HAL Id : hal-02381138, version 1



Xuesong Niu, Xingyuan Zhao, Hu Han, Abhijit Das, Antitza Dantcheva, et al.. Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention. FG 2019 - 14th IEEE International Conference on Automatic Face and Gesture Recognition, May 2019, Lille, France. ⟨hal-02381138⟩



Record views


Files downloads