Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding

Abstract : Computer vision has a great potential to help our daily lives by searching for lost keys, watering flowers or reminding us to take a pill. To succeed with such tasks, computer vision methods need to be trained from real and diverse examples of our daily dynamic scenes. While most of such scenes are not particularly exciting, they typically do not appear on YouTube, in movies or TV broadcasts. So how do we collect sufficiently many diverse but boring samples representing our lives? We propose a novel Hollywood in Homes approach to collect such data. Instead of shooting videos in the lab, we ensure diversity by distributing and crowdsourcing the whole process of video creation from script writing to video recording and annotation. Following this procedure we collect a new dataset, Charades, with hundreds of people recording videos in their own homes, acting out casual everyday activities. The dataset is composed of 9,848 annotated videos with an average length of 30 seconds, showing activities of 267 people from three continents. Each video is annotated by multiple free-text descriptions, action labels, action intervals and classes of interacted objects. In total, Charades provides 27,847 video descriptions, 66,500 temporally localized intervals for 157 action classes and 41,104 labels for 46 object classes. Using this rich data, we evaluate and provide baseline results for several tasks including action recognition and automatic description generation. We believe that the realism, diversity, and casual nature of this dataset will present unique challenges and new opportunities for computer vision community.
Type de document :
Communication dans un congrès
Computer Vision – ECCV 2016, Oct 2016, Amsterdam, Netherlands. pp.510 - 526, 2016, 〈10.1007/978-3-319-46448-0_31〉
Liste complète des métadonnées

Littérature citée [40 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01418216
Contributeur : Gul Varol <>
Soumis le : vendredi 16 décembre 2016 - 14:33:45
Dernière modification le : jeudi 11 janvier 2018 - 06:28:04
Document(s) archivé(s) le : mardi 28 mars 2017 - 00:36:38

Fichier

eccv16_charades.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Gunnar Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, et al.. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. Computer Vision – ECCV 2016, Oct 2016, Amsterdam, Netherlands. pp.510 - 526, 2016, 〈10.1007/978-3-319-46448-0_31〉. 〈hal-01418216〉

Partager

Métriques

Consultations de la notice

204

Téléchargements de fichiers

64