Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression

Antoine Deleforge 1 Radu Horaud 1, * Yoav Y. Schechner 2 Laurent Girin 3
* Corresponding author
1 PERCEPTION - Interpretation and Modelling of Images and Videos
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
3 GIPSA-CRISSP - CRISSP
GIPSA-DPC - Département Parole et Cognition
Abstract : This paper addresses the problem of localizing audio sources using binaural measurements. We propose a supervised formulation that simultaneously localizes multiple sources at different locations. The approach is intrinsically efficient because, contrary to prior work, it relies neither on source separation, nor on monaural segregation. The method starts with a training stage that establishes a locally-linear Gaussian regression model between the directional coordinates of all the sources and the auditory features extracted from binaural measurements. While fixed-length wide-spectrum sounds (white noise) are used for training to reliably estimate the model parameters, we show that the testing (localization) can be extended to variable-length sparse-spectrum sounds (such as speech), thus enabling a wide range of realistic applications. Indeed, we demonstrate that the method can be used for audio-visual fusion, namely to map speech signals onto images and hence to spatially align the audio and visual modalities, thus enabling to discriminate between speaking and non-speaking faces. We release a novel corpus of real-room recordings that allow quantitative evaluation of the co-localization method in the presence of one or two sound sources. Experiments demonstrate increased accuracy and speed relative to several state-of-the-art methods.
Complete list of metadatas

Cited literature [35 references]  Display  Hide  Download


https://hal.inria.fr/hal-01112834
Contributor : Team Perception <>
Submitted on : Wednesday, March 4, 2015 - 3:38:05 PM
Last modification on : Wednesday, April 11, 2018 - 1:58:34 AM
Long-term archiving on : Friday, June 5, 2015 - 11:05:37 AM

Files

submission_taslp14F.pdf
Files produced by the author(s)

Identifiers

Citation

Antoine Deleforge, Radu Horaud, Yoav Y. Schechner, Laurent Girin. Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression. IEEE Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, 2015, 23 (4), pp.718-731. ⟨10.1109/TASLP.2015.2405475⟩. ⟨hal-01112834v2⟩

Share

Metrics

Record views

1176

Files downloads

837