A flexible model for training action localization with varying levels of supervision - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

A flexible model for training action localization with varying levels of supervision

Résumé

Spatio-temporal action detection in videos is typically addressed in a fully-supervised setup with manual annotation of training videos required at every frame. Since such annotation is extremely tedious and prohibits scalability, there is a clear need to minimize the amount of manual supervision. In this work we propose a unifying framework that can handle and combine varying types of less-demanding weak supervision. Our model is based on discriminative clustering and integrates different types of supervision as constraints on the optimization. We investigate applications of such a model to training setups with alternative supervisory signals ranging from video-level class labels to the full per-frame annotation of action bounding boxes. Experiments on the challenging UCF101-24 and DALY datasets demonstrate competitive performance of our method at a fraction of supervision used by previous methods. The flexibility of our model enables joint learning from data with different levels of annotation. Experimental results demonstrate a significant gain by adding a few fully supervised examples to otherwise weakly labeled videos.

Dates et versions

hal-01937002 , version 1 (27-11-2018)

Identifiants

Citer

Guilhem Chéron, Jean-Baptiste Alayrac, Ivan Laptev, Cordelia Schmid. A flexible model for training action localization with varying levels of supervision. NIPS 2018 - 32nd Conference on Neural Information Processing Systems, Dec 2018, Montréal, Canada. pp.1-17. ⟨hal-01937002⟩
183 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More