Creating Segments and Effects on Comics by Clustering Gaze Data - Archive ouverte HAL Access content directly
Journal Articles ACM Transactions on Multimedia Computing, Communications and Applications Year : 2017

Creating Segments and Effects on Comics by Clustering Gaze Data

(1) , (1) , (1) , (2, 3) , (1) , (1)
1
2
3

Abstract

Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations.
Not file

Dates and versions

hal-01650400 , version 1 (28-11-2017)

Identifiers

Cite

Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea, et al.. Creating Segments and Effects on Comics by Clustering Gaze Data. ACM Transactions on Multimedia Computing, Communications and Applications, 2017, 13 (3), pp.1 - 23. ⟨10.1145/3078836⟩. ⟨hal-01650400⟩
86 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More