Skip to Main content Skip to Navigation
Conference papers

Joint Attention for Automated Video Editing

Abstract : Joint attention refers to the shared focal points of attention for occupants in a space. In this work, we introduce a computational definition of joint attention for the automated editing of meetings in multi-camera environments from the AMI corpus. Using extracted head pose and individual headset amplitude as features, we developed three editing methods: (1) a naive audio-based method that selects the camera using only the headset input, (2) a rule-based edit that selects cameras at a fixed pacing using pose data, and (3) an editing algorithm using LSTM (Long-short term memory) learned joint-attention from both pose and audio data, trained on expert edits. The methods are evaluated qualitatively against the human edit, and quantitatively in a user study with 22 participants. Results indicate that LSTM-trained joint attention produces edits that are comparable to the expert edit, offering a wider range of camera views than audio, while being more generalizable as compared to rule-based methods.
Complete list of metadata

Cited literature [32 references]  Display  Hide  Download
Contributor : Hui-Yin Wu Connect in order to contact the contributor
Submitted on : Friday, October 9, 2020 - 10:29:12 AM
Last modification on : Friday, October 9, 2020 - 10:36:11 AM
Long-term archiving on: : Sunday, January 10, 2021 - 6:08:05 PM


Publisher files allowed on an open archive




Hui-Yin Wu, Trevor Santarra, Michael Leece, Rolando Vargas, Arnav Jhala. Joint Attention for Automated Video Editing. IMX 2020 - ACM International Conference on Interactive Media Experiences, Jun 2020, Barcelona, Spain. pp.55-64, ⟨10.1145/3391614.3393656⟩. ⟨hal-02960390⟩



Record views


Files downloads