Explicit modeling of human-object interactions in realistic videos

Alessandro Prest 1, 2 Vittorio Ferrari 2 Cordelia Schmid 1
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We introduce an approach for learning human actions as interactions between persons and objects in realistic videos. Previous work typically represents actions with low-level features such as image gradients or optical flow. In contrast, we explicitly localize in space and track over time both the object and the person, and represent an action as the trajectory of the object wrt to the person position. Our approach relies on state-of-the-art techniques for human detection [32], object detection [10], and tracking [39]. We show that this results in human and object tracks of sufficient quality to model and localize human-object interactions in realistic videos. Our human-object interaction features capture the relative trajectory of the object wrt the human. Experimental results on the Coffee & Cigarettes dataset [25], the video dataset of [19] and the Rochester Daily Activities dataset [29] show that (i) our explicit human-object model is an informative cue for action recognition; (ii) it is complementary to traditional low-level descriptors such as 3D-HOG [23] extracted over human tracks. When show that combining our human-object interaction features with 3D-HOG improves over their individual performance as well as over the state-of-the-art [23], [29].
Type de document :
Article dans une revue
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2013, 35 (4), pp.835-848. 〈10.1109/TPAMI.2012.175〉
Liste complète des métadonnées

Littérature citée [13 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00720847
Contributeur : Alessandro Prest <>
Soumis le : mercredi 25 juillet 2012 - 23:36:27
Dernière modification le : mercredi 11 avril 2018 - 01:58:30
Document(s) archivé(s) le : vendredi 16 décembre 2016 - 03:13:02

Fichiers

Identifiants

Collections

Citation

Alessandro Prest, Vittorio Ferrari, Cordelia Schmid. Explicit modeling of human-object interactions in realistic videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2013, 35 (4), pp.835-848. 〈10.1109/TPAMI.2012.175〉. 〈hal-00720847〉

Partager

Métriques

Consultations de la notice

481

Téléchargements de fichiers

887