Skip to Main content Skip to Navigation

Direct Manipulation for Musical Interfaces Based on Visual Software: Design and Evaluation

Jeronimo Barbosa 1, 2 
1 IDMIL - Input Devices and Music Interaction Laboratory
CIRMMT - Centre Interdisciplinaire de Recherche en Musique, Médias et Technologie [Montréal]
2 LOKI - Technology and knowledge for interaction
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
Abstract : With the popularization of mobile computing and software-distribution platforms like the Apple Store and Google Play, a plethora of remarkable software-based musical interfaces with dynamic visual feedback are being designed today. Recent examples such as the Loopy, the Borderlands, the Ocarina illustrate that powerful visual interfaces are capable of captivating amateur and expert musicians alike by making music manipulation more direct and straightforward. However, while successful examples are available, effectively designing such direct interfaces remains challenging, typically proceeding “as more art than science”. For example, unlike acoustic instruments, sound production is physically decoupled from input controls, yielding infinite possi- bilities for mapping. Moreover, increasing ease-of-use for novices can lead to toy applications that disinterest experts. Finally, analyzing successful crafts remain unusual, and little music interface research is being carried to better understand these interfaces. This thesis attempts to understand these musical interfaces better, aiming at providing empir- ical evidence to inform their design and evaluation. Borrowing from human-computer interaction research, I hypothesize that one effective strategy concerns adopting an interaction model known as direct manipulation–characterized by continuous visual representation of the object of interest, physical actions to manipulate the visual representations, rapid visual feedback, and incremental and easily reversible actions. To investigate this hypothesis, I report exploratory case studies where direct manipulation is designed and evaluated in three specific contexts of music tools: a) live looping tools; b) tools for creating interactive artistic installations; and c) music authoring tools. As a result, I introduce three novel software music tools: Voice Reaping Machine; ZenStates; and StateSynth. Finally, I present a set of interface design strategies and evaluation approaches for effectively designing direct manipulation in musical interfaces based on visuals.
Complete list of metadata
Contributor : Stéphane Huot Connect in order to contact the contributor
Submitted on : Monday, January 13, 2020 - 1:43:33 PM
Last modification on : Sunday, June 26, 2022 - 1:12:57 AM
Long-term archiving on: : Tuesday, April 14, 2020 - 4:23:53 PM


Files produced by the author(s)


  • HAL Id : tel-02436801, version 1



Jeronimo Barbosa. Direct Manipulation for Musical Interfaces Based on Visual Software: Design and Evaluation. Human-Computer Interaction [cs.HC]. McGill University, 2019. English. ⟨tel-02436801⟩



Record views


Files downloads