Designing Interactive Spatiotemporal Visualizations to Enhance Movie Browsing

. This paper presents a case study on the design of spatiotemporal interactive visualizations of movies, both collections and contents, to provide enhanced support for conveying meaning and for browsing, targeting casual and professional users, with encouraging results for future research and adoption.


Introduction
Movies, with their rich contents conveyed in images, text, music and narration, along time, tell us stories of different places and have great emotional impact on us.Moreover, technological advances are making available huge amounts of movies over the years, on the Internet and interactive TV.However, the richness that makes these movies so interesting and accessible, comes with a challenging complexity, highlighting the need for new and powerful ways to access, browse, and view them, and interactive visualization can play an important role.In spite of the several contributions to this field, reviewed in previous publications [1,2], most approaches and services for movies and video access do not fully support this richness, and the approaches to visualization in time and space [3] do not usually address video and movies.In this paper, we present a case study on the spatiotemporal interactive visualizations that we designed to help fulfill this need, both for movie collections and contents, as an extension to our previous work on: MovieClouds [4], an interactive web application that allows to browse movies based on their content, mainly audio and subtitles, with an emotional perspective; and Sight Surfers [5], an interactive web application for sharing and browsing georeferenced 360º interactive videos, in the context of the VIRUS and ImTV projects.The proposed visualizations contribute with visual, temporal and spatial dimensions, allowing to access movies released over time, and enriching the representations and access based on different aspects of their contents, along time and space.This approach [1,2] has been refined with the ultimate goal of providing effective overviews and browsing mechanisms that may provide insights in analytical or more ludic uses, and its recent evaluation had very encouraging results towards their development and inclusion in real uses.

Goals
In the design of the interactive visualizations we aimed at their effectivenes in showing meaning; their ability in providing good usability and user experience; and their usefulness and adequacy to be adopted by casual users and professionals in real scenarios.The evaluation assessed this, following the order of these research questions: RQ1.Representations: are they intuitive and clear in reflecting meaning?RQ2.Visualizations from collection to contents: are they able to effectively show the first level of information in a glance while guiding the user in the interaction towards a goal (learnability)?RQ3.Navigation: are visualizations an added value to explore and access movies?RQ4.Target: which target users take the most of visualizations?

Spatiotemporal Visualizations: Design and Evaluation
This section presents an overview of the main visualizations designed (Fig. 1) and some highlights about their recent evaluation, in order to provide insights and lessons to inform future research and developments.It was a Task-oriented evaluation focusing on the Observation of users performing the tasks, their hesitations and errors, and semi-structured Interviews focusing on perceived usefulness, ease of use and satisfaction (USE) with 1-5 scales, and ergonomic, hedonic and appeal Quality aspects [6], while providing comments as qualitative feedback, in sessions lasting on average one hour per participant.20 participants aged 22-64, both from general public and professionals (e.g.movie directors and editors, film critics, and sound technicians).
Visualizations go from movie collections (Fig. 1 top) overviews (b-f, 2abc) or trajectories (3abc), to individual movie contents details (bottom) (4 and 5): Time: the user can observe quantities of movies released over a year by genre (1a-b), and ask for details through titles by List (1c), where the most rated movies are highlighted; or by Spots (1d), each spot corresponding to one genre of the selected movie (half dimension, e.g., corresponds to a movie with two genres).These visualizations proved effective with almost all the participants accomplishing tasks such as locating the month with more movies released.The lower frequencies are the most difficult to perceive, but this issue is easily solved by zooming in.In (1e) users can compare movies in order to know about their images, speech, audio amplitude and genre, represented in circles that reflect the duration, being possible to disclose the title, plot, images of a requested moment of the movie (by hover), or the scene playing (by click).Many people would like to have this visualization available (M:3.8; Std:1.2) and most of them considered it more suited for exploration.A movie director called it an "Evocation of the film" due to its ability to reflect emotions.In general, the participants considered the visualization a good way to find a movie to watch, and used Supporting, Interesting, Pleasant and Comprehensible [7] to characterize it.When compared with Netflix, this visualization was considered more emotional and ludic although the traditional cover image was referred to complement information.Visualization in figure (1f) seemed clear since almost all the participants could identify the most dynamic movie.It was considered less innovative than the previous visualization for its resemblance to a bar chart and more suited for movie professionals due to its perceived accuracy, reflecting the "mood" of the movie and its structure.Space: the amounts of genres released through location are represented (2a), being exemplified the detail of the romantic movies (2b) and choosing one to watch (2b-c).These visualizations were highly rated since genre and maps are commonly used.Spatiotemporal: after being aware of the country with more movies shot (3a), one area can be zoomed in towards the more shot (3b), and detailed to the more dated (brownish) and slower (higher) (3c) trajectories.The latter visualization caused some strangeness due to its novelty, but the results were still encouraging (M: 3.8).
Movie contents: show the properties of a whole movie (4) through circular elements that represent properties over time/duration (motion, colors, scene length, audio, speech, expressed and felt emotions) with the selected property highlighted as a tag cloud in the center.The user drags the circular elements that rotate in synchrony with the timeline.On the right, current scene is playing, synchronized with corresponding trajectory in a map, that can be zoomed.This visualization revealed very good results with professional users more determined to use it.The possibilities both to hide and turn visible the tracks of information, and to change properties were strongly appreciated.Visualization (5) represents one trajectory through image, audio amplitude, speech and other videos shot in that area allowing to play video at the clicked position in the trajectory.In the e.g.Before Sunrise movie in Vienna.It was very appreciated and understood by the participants.The preferred visualizations were 1e, followed by 4, and 5, reflecting interest in visualizing and exploring content in richer ways.

Lessons Learned
The results were very encouraging since all visualizations were appreciated and most proposed tasks were readily accomplished.Regarding the 4 research questions: RQ1.Representations were easy to understand, e.g.users are able to relate and compare properties among scenes.RQ2.Visualizations show in a glance what they represent; and allow RQ3.Navigation to access rich information to be explored at different levels of detail, in time and space, e.g.enter one scene and explore the location where the movie happens.They were considered an added value by all the participants, being highlighted the aesthetics and ludic way of interacting.RQ4.Casual uses were the most referred, although professionals have mentioned their interest in this kind of visualizations.

Fig. 1 .
Fig. 1.Interactive Spatiotemporal Visualizations of Movies.All visualizations allow to access more information; and to navigate, as shown by the arrows.