Skip to Main content Skip to Navigation
Journal articles

Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical Background

Wanyu Liu 1 Michelle Magalhaes 1 Wendy Mackay 2 Frédéric Bevilacqua 1 Michel Beaudouin-Lafon 2 
2 EX-SITU - Extreme Situated Interaction
Inria Saclay - Ile de France, LISN - Laboratoire Interdisciplinaire des Sciences du Numérique, IaH - Interaction avec l'Humain
Abstract : With the increasing interest in movement sonification and expressive gesture-based interaction, it is important to understand which factors contribute to movement learning and how. We explore the effects of movement sonification and users' musical background on motor variability in complex gesture learning. We contribute an empirical study in which musicians and non-musicians learn two gesture sequences over three days, with and without movement sonification. Results show the interlaced interaction effects of these factors and how they unfold in the 3-day learning process. For gesture 1 that is fast and dynamic with a direct "action-sound" sonification, movement sonification induces higher variability for both musicians and non-musicians on day 1. While musicians reduce this variability to a similar level as no auditory feedback condition on day 2 and day 3, non-musicians remain to have significantly higher variability. Across 3 days, musicians also have significantly lower variability than non-musicians. For gesture 2 that is slow and smooth with an "action-music" metaphor, there are virtually no effects. Based on these findings, we recommend future studies to take into account participants' musical background; consider longitudinal study to examine these effects on complex gestures; and be aware when interpreting the results given a specific design of gesture and sound.
Document type :
Journal articles
Complete list of metadata
Contributor : Michel Beaudouin-Lafon Connect in order to contact the contributor
Submitted on : Wednesday, March 9, 2022 - 8:18:01 PM
Last modification on : Friday, August 5, 2022 - 9:27:34 AM


Files produced by the author(s)



Wanyu Liu, Michelle Magalhaes, Wendy Mackay, Frédéric Bevilacqua, Michel Beaudouin-Lafon. Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical Background. ACM Transactions on Applied Perception, Association for Computing Machinery, 2022, 19 (1), pp.1 - 21. ⟨10.1145/3482967⟩. ⟨hal-03603501⟩



Record views


Files downloads