Creating Segments and Effects on Comics by Clustering Gaze Data

Abstract : Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations.
Document type :
Journal articles
Complete list of metadatas

https://hal.inria.fr/hal-01650400
Contributor : Olivier Le Meur <>
Submitted on : Tuesday, November 28, 2017 - 1:44:24 PM
Last modification on : Friday, September 13, 2019 - 9:50:02 AM

Identifiers

Citation

Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea, et al.. Creating Segments and Effects on Comics by Clustering Gaze Data. ACM Transactions on Multimedia Computing, Communications and Applications, Association for Computing Machinery, 2017, 13 (3), pp.1 - 23. ⟨10.1145/3078836⟩. ⟨hal-01650400⟩

Share

Metrics

Record views

209