PoTion: Pose MoTion Representation for Action Recognition

Abstract : Most state-of-the-art methods for action recognition rely on a two-stream architecture that processes appearance and motion independently. In this paper, we claim that considering them jointly offers rich information for action recognition. We introduce a novel representation that gracefully encodes the movement of some semantic keypoints. We use the human joints as these keypoints and term our Pose moTion representation PoTion. Specifically, we first run a state-of-the-art human pose estimator [4] and extract heatmaps for the human joints in each frame. We obtain our PoTion representation by temporally aggregating these probability maps. This is achieved by 'colorizing' each of them depending on the relative time of the frames in the video clip and summing them. This fixed-size representation for an entire video clip is suitable to classify actions using a shallow convolutional neural network. Our experimental evaluation shows that PoTion outper-forms other state-of-the-art pose representations [6, 48]. Furthermore, it is complementary to standard appearance and motion streams. When combining PoTion with the recent two-stream I3D approach [5], we obtain state-of-the-art performance on the JHMDB, HMDB and UCF101 datasets.
Type de document :
Communication dans un congrès
CVPR 2018 - IEEE Conference on Computer Vision and Pattern Recognition, Jun 2018, Salt Lake City, United States. IEEE, pp.1-10, 2018
Liste complète des métadonnées

Littérature citée [48 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01764222
Contributeur : Xavier Martin <>
Soumis le : mercredi 11 avril 2018 - 16:52:32
Dernière modification le : mercredi 18 avril 2018 - 11:10:10

Fichier

PoTion Pose MoTion Representat...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01764222, version 1

Collections

Citation

Vasileios Choutas, Philippe Weinzaepfel, Jérôme Revaud, Cordelia Schmid. PoTion: Pose MoTion Representation for Action Recognition. CVPR 2018 - IEEE Conference on Computer Vision and Pattern Recognition, Jun 2018, Salt Lake City, United States. IEEE, pp.1-10, 2018. 〈hal-01764222〉

Partager

Métriques

Consultations de la notice

1465

Téléchargements de fichiers

2325