Creating Segments and Effects on Comics by Clustering Gaze Data

Abstract : Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations.
Type de document :
Article dans une revue
ACM Transactions on Multimedia Computing, Communications and Applications, Association for Computing Machinery, 2017, 13 (3), pp.1 - 23. 〈10.1145/3078836〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01650400
Contributeur : Olivier Le Meur <>
Soumis le : mardi 28 novembre 2017 - 13:44:24
Dernière modification le : vendredi 16 novembre 2018 - 01:28:23

Identifiants

Citation

Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea, et al.. Creating Segments and Effects on Comics by Clustering Gaze Data. ACM Transactions on Multimedia Computing, Communications and Applications, Association for Computing Machinery, 2017, 13 (3), pp.1 - 23. 〈10.1145/3078836〉. 〈hal-01650400〉

Partager

Métriques

Consultations de la notice

122