Evaluation of deep pose detectors for automatic analysis of film style - Archive ouverte HAL Access content directly
Conference Papers Proceedings of the ACM on Human-Computer Interaction Year : 2022

Evaluation of deep pose detectors for automatic analysis of film style

Abstract

Identifying human characters and how they are portrayed on-screen is inherently linked to how we perceive and interpret the story and artistic value of visual media. Building computational models sensible towards story will thus require a formal representation of the character. Yet this kind of data is complex and tedious to annotate on a large scale. Human pose estimation (HPE) can facilitate this task, to identify features such as position, size, and movement that can be transformed into input to machine learning models, and enable higher artistic and storytelling interpretation. However, current HPE methods operate mainly on non-professional image content, with no comprehensive evaluation of their performance on artistic film. Our goal in this paper is thus to evaluate the performance of HPE methods on artistic film content. We first propose a formal representation of the character based on cinematography theory, then sample and annotate 2700 images from three datasets with this representation, one of which we introduce to the community. An in-depth analysis is then conducted to measure the general performance of two recent HPE methods on metrics of precision and recall for character detection , and to examine the impact of cinematographic style. From these findings, we highlight the advantages of HPE for automated film analysis, and propose future directions to improve their performance on artistic film content.
Fichier principal
Vignette du fichier
WICED2022_paper_1166.pdf (5.19 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03634624 , version 1 (07-04-2022)

Identifiers

  • HAL Id : hal-03634624 , version 1

Cite

Hui-Yin Wu, Luan Nguyen, Yoldoz Tabei, Lucile Sassatelli. Evaluation of deep pose detectors for automatic analysis of film style. 10th Eurographics Workshop on Intelligent Cinematography and Editing, Apr 2022, Reims, France. ⟨hal-03634624⟩
128 View
65 Download

Share

Gmail Facebook Twitter LinkedIn More