Action Recognition by Dense Trajectories

Heng Wang 1, 2, * Alexander Kläser 1 Cordelia Schmid 1 Liu Cheng-Lin 2
* Corresponding author
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : Feature trajectories have shown to be efficient for representing videos. Typically, they are extracted using the KLT tracker or matching SIFT descriptors between frames. However, the quality as well as quantity of these trajectories is often not sufficient. Inspired by the recent success of dense sampling in image classification, we propose an approach to describe videos by dense trajectories. We sample dense points from each frame and track them based on displacement information from a dense optical flow field. Given a state-of-the-art optical flow algorithm, our trajectories are robust to fast irregular motions as well as shot boundaries. Additionally, dense trajectories cover the motion information in videos well. We, also, investigate how to design descriptors to encode the trajectory information. We introduce a novel descriptor based on motion boundary histograms, which is robust to camera motion. This descriptor consistently outperforms other state-of-the-art descriptors, in particular in uncontrolled realistic videos. We evaluate our video description in the context of action classification with a bag-of-features approach. Experimental results show a significant improvement over the state of the art on four datasets of varying difficulty, i.e. KTH, YouTube, Hollywood2 and UCF sports.
Document type :
Conference papers
CVPR 2011 - IEEE Conference on Computer Vision & Pattern Recognition, Jun 2011, Colorado Springs, United States. IEEE, pp.3169-3176, 2011, 〈10.1109/CVPR.2011.5995407〉
Liste complète des métadonnées

Cited literature [35 references]  Display  Hide  Download

https://hal.inria.fr/inria-00583818
Contributor : Thoth Team <>
Submitted on : Thursday, April 7, 2011 - 10:13:18 AM
Last modification on : Friday, November 3, 2017 - 10:24:06 PM
Document(s) archivé(s) le : Friday, July 8, 2011 - 2:35:35 AM

File

wang_cvpr11.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Heng Wang, Alexander Kläser, Cordelia Schmid, Liu Cheng-Lin. Action Recognition by Dense Trajectories. CVPR 2011 - IEEE Conference on Computer Vision & Pattern Recognition, Jun 2011, Colorado Springs, United States. IEEE, pp.3169-3176, 2011, 〈10.1109/CVPR.2011.5995407〉. 〈inria-00583818〉

Share

Metrics

Record views

5231

Files downloads

28581