Finding Actors and Actions in Movies

Piotr Bojanowski 1, 2 Francis Bach 1, 3 Ivan Laptev 1, 2 Jean Ponce 1, 2 Cordelia Schmid 4 Josef Sivic 1, 2
1 LIENS - Laboratoire d'informatique de l'école normale supérieure
2 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
3 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
4 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We address the problem of learning a joint model of actors and actions in movies using weak supervision provided by scripts. Specifically, we extract actor/action pairs from the script and use them as constraints in a discriminative clustering framework. The corresponding optimization problem is formulated as a quadratic program under linear constraints. People in video are represented by automatically extracted and tracked faces together with corresponding motion features. First, we apply the proposed framework to the task of learning names of characters in the movie and demonstrate significant improvements over previous methods used for this task. Second, we explore the joint actor/action constraint and show its advantage for weakly supervised action learning. We validate our method in the challenging setting of localizing and recognizing characters and their actions in feature length movies Casablanca and American Beauty.
Type de document :
Communication dans un congrès
ICCV 2013 - IEEE International Conference on Computer Vision, Dec 2013, Sydney, Australia. IEEE, pp.2280-2287, 2013, 〈10.1109/ICCV.2013.283〉
Liste complète des métadonnées

Littérature citée [25 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-00904991
Contributeur : Piotr Bojanowski <>
Soumis le : vendredi 15 novembre 2013 - 16:26:19
Dernière modification le : lundi 17 décembre 2018 - 11:22:02
Document(s) archivé(s) le : lundi 17 février 2014 - 16:46:01

Fichier

joint_learning_names.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

INRIA | LJK_GI | LJK_GI_LEAR | LJK | UGA | ENS-PARIS | QUAERO | PSL | INRIA-ROCQ

Citation

Piotr Bojanowski, Francis Bach, Ivan Laptev, Jean Ponce, Cordelia Schmid, et al.. Finding Actors and Actions in Movies. ICCV 2013 - IEEE International Conference on Computer Vision, Dec 2013, Sydney, Australia. IEEE, pp.2280-2287, 2013, 〈10.1109/ICCV.2013.283〉. 〈hal-00904991〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

1284

Téléchargements de fichiers

711

Contact


archive-ouverte@inria.fr