Acoustic Space Learning for Sound-Source Separation and Localization on Binaural Manifolds

Antoine Deleforge 1, * Florence Forbes 2, * Radu Horaud 1, *
* Auteur correspondant
1 PERCEPTION - Interpretation and Modelling of Images and Videos
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
2 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : In this paper we address the problems of modeling the acoustic space generated by a full-spectrum sound source and of using the learned model for the localization and separation of multiple sources that simultaneously emit sparse-spectrum sounds. We lay theoretical and methodological grounds in order to introduce the binaural manifold paradigm. We perform an in-depth study of the latent low-dimensional structure of the high-dimensional interaural spectral data, based on a corpus recorded with a human-like audiomotor robot head. A non-linear dimensionality reduction technique is used to show that these data lie on a two-dimensional (2D) smooth manifold parameterized by the motor states of the listener, or equivalently, the sound source directions. We propose a probabilistic piecewise affine mapping model (PPAM) specifically designed to deal with high-dimensional data exhibiting an intrinsic piecewise linear structure. We derive a closed-form expectation-maximization (EM) procedure for estimating the model parameters, followed by Bayes inversion for obtaining the full posterior density function of a sound source direction. We extend this solution to deal with missing data and redundancy in real world spectrograms, and hence for 2D localization of natural sound sources such as speech. We further generalize the model to the challenging case of multiple sound sources and we propose a variational EM framework. The associated algorithm, referred to as variational EM for source separation and localization (VESSL) yields a Bayesian estimation of the 2D locations and time-frequency masks of all the sources. Comparisons of the proposed approach with several existing methods reveal that the combination of acoustic-space learning with Bayesian inference enables our method to outperform state-of-the-art methods.
Liste complète des métadonnées


https://hal.inria.fr/hal-00960796
Contributeur : Team Perception <>
Soumis le : mercredi 19 mars 2014 - 09:11:19
Dernière modification le : mercredi 14 décembre 2016 - 01:07:11

Fichiers

submission-ijns13.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Antoine Deleforge, Florence Forbes, Radu Horaud. Acoustic Space Learning for Sound-Source Separation and Localization on Binaural Manifolds. International Journal of Neural Systems, World Scientific Publishing, 2015, 25 (1), 21p. <http://www.worldscientific.com/doi/abs/10.1142/S0129065714400036>. <10.1142/S0129065714400036>. <hal-00960796>

Partager

Métriques

Consultations de
la notice

526

Téléchargements du document

299