Towards Weakly-Supervised Action Localization

Philippe Weinzaepfel 1 Xavier Martin 1 Cordelia Schmid 1
1 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : This paper presents a novel approach for weakly-supervised action localization, i.e., that does not require per-frame spatial annotations for training. We first introduce an effective method for extracting human tubes by combining a state-of-the-art human detector with a tracking-by-detection approach. Our tube extraction leverages the large amount of annotated humans available today and outperforms the state of the art by an order of magnitude: with less than 5 tubes per video, we obtain a recall of 95% on the UCF-Sports and J-HMDB datasets. Given these human tubes, we perform weakly-supervised selection based on multi-fold Multiple Instance Learning (MIL) with improved dense trajectories and achieve excellent results. We obtain a mAP of 84% on UCF-Sports, 54% on J-HMDB and 45% on UCF-101, which outperforms the state of the art for weakly-supervised action localization and is close to the performance of the best fully-supervised approaches. The second contribution of this paper is a new realistic dataset for action localization, named DALY (Daily Action Localization in YouTube). It contains high quality temporal and spatial annotations for 10 actions in 31 hours of videos (3.3M frames), which is an order of magnitude larger than standard action localization datasets. On the DALY dataset, our tubes have a spatial recall of 82%, but the detection task is extremely challenging, we obtain 10.8% mAP.
Type de document :
Pré-publication, Document de travail
2016
Liste complète des métadonnées



https://hal.inria.fr/hal-01317558
Contributeur : Thoth Team <>
Soumis le : mercredi 18 mai 2016 - 15:18:44
Dernière modification le : mardi 26 juillet 2016 - 15:49:45
Document(s) archivé(s) le : jeudi 17 novembre 2016 - 13:26:16

Fichiers

weak_action.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01317558, version 1
  • ARXIV : 1605.05197

Collections

Citation

Philippe Weinzaepfel, Xavier Martin, Cordelia Schmid. Towards Weakly-Supervised Action Localization. 2016. <hal-01317558>

Partager

Métriques

Consultations de
la notice

501

Téléchargements du document

1465