Explicit modeling of human-object interactions in realistic videos

Alessandro Prest 1, 2 Vittorio Ferrari 2 Cordelia Schmid 1
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We introduce an approach for learning human actions as interactions between persons and objects in realistic videos. Previous works typically represent actions with low-level features such as image gradients or optical flow. In contrast, we explicitly localize in space and track over time both the object and the person, and represent an action as the trajectory of the object wrt to the person position. Our approach relies on state-of-the-art approaches for human [32] and object detection [10] as well as tracking [39]. We show that this results in human and object tracks of sufficient quality to model and localize human-object interactions in realistic videos. Our human-object interaction features capture relative trajectory of the object wrt the human. Experimental results on the Coffee & Cigarettes [25], the video dataset of [19] and the Rochester Daily Activities dataset [29] show that (i) our explicit human-object model is an informative cue for action recognition; (ii) it is complementary to traditional low-level descriptors such as 3D-HOG extracted over human tracks. When combining our human-object interaction features with 3D-HOG features [23], we show to improve over their separate performance as well as over the state of the art.
Document type :
Reports
[Technical Report] RT-0411, 2011
Liste complète des métadonnées


https://hal.inria.fr/inria-00626929
Contributor : Alessandro Prest <>
Submitted on : Thursday, May 10, 2012 - 12:16:04 PM
Last modification on : Saturday, September 17, 2016 - 1:36:53 AM
Document(s) archivé(s) le : Thursday, December 15, 2016 - 5:39:53 AM

File

CC.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00626929, version 4

Collections

Citation

Alessandro Prest, Vittorio Ferrari, Cordelia Schmid. Explicit modeling of human-object interactions in realistic videos. [Technical Report] RT-0411, 2011. <inria-00626929v4>

Share

Metrics

Record views

754

Document downloads

373