Evaluation of local spatio-temporal features for action recognition

Heng Wang; Muhammad Muneeb Ullah; Alexander Klaser; Ivan Laptev; Cordelia Schmid

inria-00439769, version 1

Evaluation of local spatio-temporal features for action recognition

Heng Wang ^1, 2, Muhammad Muneeb Ullah ()^a^, 3, Alexander Klaser ()^b^, 1, Ivan Laptev ()^a^, 3, Cordelia Schmid ()^a^, 1

BMVC 2009 - British Machine Vision Conference (2009)

Abstract: Local space-time features have recently become a popular video representation for action recognition. Several methods for feature localization and description have been proposed in the literature and promising recognition results were demonstrated for a number of action classes. The comparison of existing methods, however, is often limited given the different experimental settings used. The purpose of this paper is to evaluate and compare previously proposed space-time features in a common experimental setup. In particular, we consider four different feature detectors and six local feature descriptors and use a standard bag-of-features SVM approach for action recognition. We investigate the performance of these methods on a total of 25 action classes distributed over three datasets with varying difficulty. Among interesting conclusions, we demonstrate that regular sampling of space-time features consistently outperforms all tested space-time interest point detectors for human actions in realistic settings. We also demonstrate a consistent ranking for the majority of methods over different datasets and discuss their advantages and limitations.

a – INRIA
b – CNRS
1: LEAR (INRIA Grenoble Rhône-Alpes / LJK Laboratoire Jean Kuntzmann)
CNRS : UMR5527 – INRIA – Laboratoire Jean Kuntzmann – Université Joseph Fourier - Grenoble I – Institut National Polytechnique de Grenoble (INPG)
2: Pattern (LIAMA)
Centre de coopération internationale en recherche agronomique pour le développement [CIRAD] – CNRS – Institut national de la recherche agronomique (INRA) – INRIA – Chinese Academy of Science (CAS) – Institute of Automation, Chinese Academy of Sciences
3: VISTAS (INRIA - IRISA)
INRIA – Institut National des Sciences Appliquées (INSA) - Rennes – CNRS : UMR6074 – Université de Rennes 1 – École normale supérieure de Cachan - ENS Cachan

inria-00439769, version 1
http://hal.inria.fr/inria-00439769
oai:hal.inria.fr:inria-00439769
From: Team Lear <>
Submitted on: Friday, 8 April 2011 14:18:13
Updated on: Tuesday, 10 April 2012 11:29:25

See detailed view

Associated documents

PDF :

Annexes :

VIDEO :

Export

Bibtex EndNote TEI RefWorks

%% inria-00439769, version 1
%% http://hal.inria.fr/inria-00439769
@inproceedings{wang:inria-00439769,
    hal_id = {inria-00439769},
    url = {http://hal.inria.fr/inria-00439769},
    title = {{Evaluation of local spatio-temporal features for action recognition}},
    author = {Wang, Heng and Ullah, Muhammad Muneeb and Klaser, Alexander and Laptev, Ivan and Schmid, Cordelia},
    abstract = {{Local space-time features have recently become a popular video representation for action recognition. Several methods for feature localization and description have been proposed in the literature and promising recognition results were demonstrated for a number of action classes. The comparison of existing methods, however, is often limited given the different experimental settings used. The purpose of this paper is to evaluate and compare previously proposed space-time features in a common experimental setup. In particular, we consider four different feature detectors and six local feature descriptors and use a standard bag-of-features SVM approach for action recognition. We investigate the performance of these methods on a total of 25 action classes distributed over three datasets with varying difficulty. Among interesting conclusions, we demonstrate that regular sampling of space-time features consistently outperforms all tested space-time interest point detectors for human actions in realistic settings. We also demonstrate a consistent ranking for the majority of methods over different datasets and discuss their advantages and limitations.}},
    keywords = {descriptor, evaluation, action, recognition, classification, SVM, BOW, BOF, 3D, videos, KTH, Hollywood2, Hollywood, UCF sports},
    language = {English},
    affiliation = {LEAR - INRIA Grenoble Rh{\^o}ne-Alpes / LJK Laboratoire Jean Kuntzmann , Pattern - LIAMA , VISTAS - INRIA - IRISA},
    booktitle = {{BMVC 2009 - British Machine Vision Conference}},
    address = {London, United Kingdom},
    note = {CLASS },
    audience = {international },
    year = {2009},
    month = Sep,
    pdf = {http://hal.inria.fr/inria-00439769/PDF/paper.pdf},
}

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="inproceedings"><head>Peer-reviewed conferences/proceedings</head>
<biblStruct type="inproceedings" xml:id="inria-00439769"><analytic>
<author><persName><forename>Heng</forename><surname>Wang</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Grenoble Rhône-Alpes / LJK Laboratoire Jean Kuntzmann</orgName>
<orgName type="team">LEAR</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5527</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Laboratoire Jean Kuntzmann</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut National Polytechnique de Grenoble (INPG)</orgName>
</affiliation>
<affiliation>
<orgName type="laboratory">LIAMA</orgName>
<orgName type="team">Pattern</orgName>
<country>CN</country>
<orgName type="institution">Centre de coopération internationale en recherche agronomique pour le développement [CIRAD]</orgName>
<orgName type="institution">CNRS</orgName>
<orgName type="institution">Institut national de la recherche agronomique (INRA)</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Chinese Academy of Science (CAS)</orgName>
<orgName type="institution">Institute of Automation, Chinese Academy of Sciences</orgName>
</affiliation>
</author>
<author><persName><forename>Muhammad Muneeb</forename><surname>Ullah</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA - IRISA</orgName>
<orgName type="team">VISTAS</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Institut National des Sciences Appliquées (INSA) - Rennes</orgName>
<orgName type="institution">CNRS : UMR6074</orgName>
<orgName type="institution">Université de Rennes 1</orgName>
<orgName type="institution">École normale supérieure de Cachan - ENS Cachan</orgName>
</affiliation>
</author>
<author><persName><forename>Alexander</forename><surname>Klaser</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Grenoble Rhône-Alpes / LJK Laboratoire Jean Kuntzmann</orgName>
<orgName type="team">LEAR</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5527</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Laboratoire Jean Kuntzmann</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut National Polytechnique de Grenoble (INPG)</orgName>
</affiliation>
</author>
<author><persName><forename>Ivan</forename><surname>Laptev</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA - IRISA</orgName>
<orgName type="team">VISTAS</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Institut National des Sciences Appliquées (INSA) - Rennes</orgName>
<orgName type="institution">CNRS : UMR6074</orgName>
<orgName type="institution">Université de Rennes 1</orgName>
<orgName type="institution">École normale supérieure de Cachan - ENS Cachan</orgName>
</affiliation>
</author>
<author><persName><forename>Cordelia</forename><surname>Schmid</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Grenoble Rhône-Alpes / LJK Laboratoire Jean Kuntzmann</orgName>
<orgName type="team">LEAR</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR5527</orgName>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">Laboratoire Jean Kuntzmann</orgName>
<orgName type="institution">Université Joseph Fourier - Grenoble I</orgName>
<orgName type="institution">Institut National Polytechnique de Grenoble (INPG)</orgName>
</affiliation>
</author>
<title type="main" level="a">Evaluation of local spatio-temporal features for action recognition</title></analytic><monogr><meeting>BMVC 2009 - British Machine Vision Conference</meeting><imprint><pubPlace><settlement>London</settlement><country>United Kingdom</country></pubPlace><biblScope type="publicationDate">2009-09-00</biblScope></imprint></monogr><idno type="HALid">http://hal.inria.fr/inria-00439769</idno><idno type="HALFile">http://hal.inria.fr/inria-00439769/PDF/paper.pdf</idno></biblStruct>
</listBibl>
</listBibl>