Exploiting the Complementarity of Audio and Visual Data in Multi-Speaker Tracking - Archive ouverte HAL Access content directly
Conference Papers Year : 2018

Exploiting the Complementarity of Audio and Visual Data in Multi-Speaker Tracking

(1) , (1, 2) , (1) , (1)
1
2

Abstract

Multi-speaker tracking is a central problem in human-robot interaction. In this context, exploiting auditory and visual information is gratifying and challenging at the same time. Gratifying because the complementary nature of auditory and visual information allows us to be more robust against noise and outliers than unimodal approaches. Challenging because how to properly fuse auditory and visual information for multi-speaker tracking is far from being a solved problem. In this paper we propose a probabilistic generative model that tracks multiple speakers by jointly exploiting auditory and visual features in their own representation spaces. Importantly, the method is robust to missing data and is therefore able to track even when observations from one of the modalities are absent. Quantitative and qualitative results on the AVDIAR dataset are reported.
Vignette du fichier
0504_1000.png (467.97 Ko) Télécharger le fichier Fichier principal
Vignette du fichier
ICCVW_submission.pdf (5.57 Mo) Télécharger le fichier
Vignette du fichier
0504_1000.jpg (38.89 Ko) Télécharger le fichier
Format : Figure, Image
Origin : Files produced by the author(s)
Origin : Files produced by the author(s)
Format : Figure, Image
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01577965 , version 1 (28-08-2017)

Identifiers

Cite

Yutong Ban, Laurent Girin, Xavier Alameda-Pineda, Radu Horaud. Exploiting the Complementarity of Audio and Visual Data in Multi-Speaker Tracking. ICCVW 2017 - IEEE International Conference on Computer Vision Workshops, Oct 2017, Venise, Italy. pp.446-454, ⟨10.1109/ICCVW.2017.60⟩. ⟨hal-01577965⟩
495 View
296 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More