Long-term Temporal Convolutions for Action Recognition

Gül Varol 1, 2 Ivan Laptev 1 Cordelia Schmid 2
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
2 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : Typical human actions last several seconds and exhibit characteristic spatio-temporal structure. Recent methods attempt to capture this structure and learn action representations with convolutional neural networks. Such representations, however, are typically learned at the level of a few video frames failing to model actions at their full temporal extent. In this work we learn video representations using neural networks with long-term temporal convolutions (LTC). We demonstrate that LTC-CNN models with increased temporal extents improve the accuracy of action recognition. We also study the impact of different low-level representations, such as raw values of video pixels and optical flow vector fields and demonstrate the importance of high-quality optical flow estimation for learning accurate action models. We report state-of-the-art results on two challenging benchmarks for human action recognition UCF101 (92.7%) and HMDB51 (67.2%).
Type de document :
Pré-publication, Document de travail
2016
Liste complète des métadonnées


https://hal.inria.fr/hal-01241518
Contributeur : Gul Varol <>
Soumis le : vendredi 15 avril 2016 - 16:02:41
Dernière modification le : vendredi 16 décembre 2016 - 14:49:52

Fichier

varol2016.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01241518, version 2
  • ARXIV : 1604.04494

Collections

Citation

Gül Varol, Ivan Laptev, Cordelia Schmid. Long-term Temporal Convolutions for Action Recognition. 2016. <hal-01241518>

Partager

Métriques

Consultations de
la notice

856

Téléchargements du document

1203