Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points

Fabien Baradel 1 Christian Wolf 1, 2 Julien Mille 3 Graham W. Taylor 4
1 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
2 CHROMA - Robots coopératifs et adaptés à la présence humaine en environnements dynamiques
CITI - CITI Centre of Innovation in Telecommunications and Integration of services, Inria Grenoble - Rhône-Alpes
Abstract : We propose a method for human activity recognition from RGB data that does not rely on any pose information during test time, and does not explicitly calculate pose information internally. Instead, a visual attention module learns to predict glimpse sequences in each frame. These glimpses correspond to interest points in the scene that are relevant to the classified activities. No spatial coherence is forced on the glimpse locations, which gives the attention module liberty to explore different points at each frame and better optimize the process of scrutinizing visual information. Tracking and sequentially integrating this kind of un-structured data is a challenge, which we address by separating the set of glimpses from a set of recurrent track-ing/recognition workers. These workers receive glimpses, jointly performing subsequent motion tracking and activity prediction. The glimpses are soft-assigned to the workers , optimizing coherence of the assignments in space, time and feature space using an external memory module. No hard decisions are taken, i.e. each glimpse point is assigned to all existing workers, albeit with different importance. Our methods outperform the state-of-the-art on the largest human activity recognition dataset available to-date, NTU RGB+D, and on the Northwestern-UCLA Multiview Action 3D Dataset.
Type de document :
Communication dans un congrès
CVPR 2018 - Computer Vision and Pattern Recognition, Jun 2018, Salt Lake City, United States. IEEE, pp.1-10
Liste complète des métadonnées

Littérature citée [35 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01713109
Contributeur : Christian Wolf <>
Soumis le : mercredi 25 juillet 2018 - 11:57:58
Dernière modification le : mercredi 31 octobre 2018 - 12:24:25
Document(s) archivé(s) le : vendredi 26 octobre 2018 - 13:39:18

Fichier

0182.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01713109, version 1

Collections

Citation

Fabien Baradel, Christian Wolf, Julien Mille, Graham W. Taylor. Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points. CVPR 2018 - Computer Vision and Pattern Recognition, Jun 2018, Salt Lake City, United States. IEEE, pp.1-10. 〈hal-01713109〉

Partager

Métriques

Consultations de la notice

411

Téléchargements de fichiers

45