Predicting Future Instance Segmentation by Forecasting Convolutional Features

Pauline Luc 1 Camille Couprie 1 Yann Lecun 2 Jakob Verbeek 3
1 FAIR - Facebook AI Research [Paris]
2 CIMS - Courant Institute of Mathematical Sciences [New York]
3 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : Anticipating future events is an important prerequisite towards intelligent behavior. Video forecasting has been studied as a proxy task towards this goal. Recent work has shown that to predict semantic segmentation of future frames, forecasting at the semantic level is more effective than forecasting RGB frames and then segmenting these. In this paper we consider the more challenging problem of future instance segmentation, which additionally segments out individual objects. To deal with a varying number of output labels per image, we develop a predictive model in the space of fixed-sized convolutional features of the Mask R-CNN instance segmentation model. We apply the "detection head" of Mask R-CNN on the predicted features to produce the instance segmentation of future frames. Experiments show that this approach significantly improves over strong baselines based on optical flow and repurposed instance segmentation architectures.
Keywords : Instance Segmentation Convolutional neural networks Deep learning Video Prediction
Type de document :
Communication dans un congrès
ECCV 2018 - European Conference on Computer Vision, Sep 2018, Munich, Germany. Springer, 11213, pp.593-608, Lecture Notes in Computer Science. 〈10.1007/978-3-030-01240-3_36〉
Liste complète des métadonnées
https://hal.inria.fr/hal-01757669
Contributeur : Pauline Luc <>
Soumis le : mercredi 3 octobre 2018 - 12:19:52
Dernière modification le : jeudi 7 février 2019 - 16:21:59

Fichier

luc18eccv_arxiv.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

PERSYVAL-LAB | INRIA | LJK | UGA | LJK_GI | LJK-GI-THOTH

Citation

Pauline Luc, Camille Couprie, Yann Lecun, Jakob Verbeek. Predicting Future Instance Segmentation by Forecasting Convolutional Features. ECCV 2018 - European Conference on Computer Vision, Sep 2018, Munich, Germany. Springer, 11213, pp.593-608, Lecture Notes in Computer Science. 〈10.1007/978-3-030-01240-3_36〉. 〈hal-01757669v2〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

290

Téléchargements de fichiers

437

Contact


archive-ouverte@inria.fr