View-Independent Action Recognition from Temporal Self-Similarities

Imran Junejo 1 Emilie Dexter 1 Ivan Laptev 2 Patrick Pérez 1
1 VISTA - Vision spatio-temporelle et active
IRISA - Institut de Recherche en Informatique et Systèmes Aléatoires, Inria Rennes – Bretagne Atlantique
2 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : This paper addresses recognition of human actions under view changes. We explore self-similarities of action se- quences over time and observe the striking stability of such measures across views. Building upon this key observation, we develop an action descriptor that captures the structure of temporal similarities and dissimilarities within an action sequence. Despite this temporal self-similarity descriptor not being strictly view-invariant, we provide intuition and experimental validation demonstrating its high stability under view changes. Self-similarity descriptors are also shown stable under performance variations within a class of actions, when individual speed fluctuations are ignored. If required, such fluctuations between two different instances of the same action class can be explicitly recovered with dynamic time warping, as will be demonstrated, to achieve cross-view action synchronization. More central to present work, temporal ordering of local self- similarity descriptors can simply be ignored within a bag-of- features type of approach. Sufficient action discrimination is still retained this way to build a view-independent action recognition system. Interestingly, self-similarities computed from different image features possess similar properties and can be used in a complementary fashion. Our method is simple and requires neither structure recovery nor multi-view correspondence es- timation. Instead, it relies on weak geometric properties and combines them with machine learning for efficient cross-view action recognition. The method is validated on three public datasets. It has similar or superior performance compared to related methods and it performs well even in extreme conditions such as when recognizing actions from top views while using side views only for training.
Type de document :
Article dans une revue
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2011
Liste complète des métadonnées

Littérature citée [54 références]  Voir  Masquer  Télécharger
Contributeur : Minsu Cho <>
Soumis le : mardi 16 septembre 2014 - 23:37:05
Dernière modification le : mardi 17 avril 2018 - 11:34:35
Document(s) archivé(s) le : mercredi 17 décembre 2014 - 11:56:24


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-01064695, version 1


Imran Junejo, Emilie Dexter, Ivan Laptev, Patrick Pérez. View-Independent Action Recognition from Temporal Self-Similarities. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2011. 〈hal-01064695〉



Consultations de la notice


Téléchargements de fichiers