Joint learning of object and action detectors

Vicky Kalogeiton 1, 2 Philippe Weinzaepfel 3 Vittorio Ferrari 2 Cordelia Schmid 1
1 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
2 CALVIN research group [Edinburgh]
IPAB - Institute of Perception, Action and Behaviour
Abstract : While most existing approaches for detection in videos focus on objects or human actions separately, we aim at jointly detecting objects performing actions, such as cat eating or dog jumping. We introduce an end-to-end multitask objective that jointly learns object-action relationships. We compare it with different training objectives, validate its effectiveness for detecting objects-actions in videos, and show that both tasks of object and action detection benefit from this joint learning. Moreover, the proposed architecture can be used for zero-shot learning of actions: our multitask objective leverages the commonalities of an action performed by different objects, e.g. dog and cat jumping , enabling to detect actions of an object without training with these object-actions pairs. In experiments on the A2D dataset [50], we obtain state-of-the-art results on segmentation of object-action pairs. We finally apply our multitask architecture to detect visual relationships between objects in images of the VRD dataset [24].
Type de document :
Communication dans un congrès
ICCV - IEEE International Conference on Computer Vision, Oct 2017, Venice, Italy
Liste complète des métadonnées



https://hal.inria.fr/hal-01575804
Contributeur : Thoth Team <>
Soumis le : lundi 21 août 2017 - 16:56:10
Dernière modification le : mardi 5 septembre 2017 - 08:27:00

Fichiers

main.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01575804, version 1

Collections

Citation

Vicky Kalogeiton, Philippe Weinzaepfel, Vittorio Ferrari, Cordelia Schmid. Joint learning of object and action detectors. ICCV - IEEE International Conference on Computer Vision, Oct 2017, Venice, Italy. <hal-01575804>

Partager

Métriques

Consultations de
la notice

297

Téléchargements du document

214