Deep neural network based multichannel audio source separation

Aditya Arie Nugraha 1 Antoine Liutkus 2 Emmanuel Vincent 1
1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
2 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : This chapter presents a multichannel audio source separation framework where deep neural networks (DNNs) are used to model the source spectra and combined with the classical multichannel Gaussian model to exploit the spatial information. The parameters are estimated in an iterative expectation-maximization (EM) fashion and used to derive a multichannel Wiener filter. Different design choices and their impact on the performance are discussed. They include the cost functions for DNN training, the number of parameter updates, the use of multiple DNNs, and the use of weighted parameter updates. Finally, we present its application to a speech enhancement task and a music separation task. The experimental results show the benefit of the multichannel DNN-based approach over a single-channel DNN-based approach and the multichannel nonnegative matrix factorization based iterative EM framework.
Document type :
Book sections
Liste complète des métadonnées

Cited literature [72 references]  Display  Hide  Download

https://hal.inria.fr/hal-01633858
Contributor : Aditya Arie Nugraha <>
Submitted on : Monday, November 13, 2017 - 2:30:56 PM
Last modification on : Wednesday, April 3, 2019 - 1:23:11 AM
Document(s) archivé(s) le : Wednesday, February 14, 2018 - 2:00:11 PM

File

nugraha_book18.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01633858, version 1

Citation

Aditya Arie Nugraha, Antoine Liutkus, Emmanuel Vincent. Deep neural network based multichannel audio source separation. Audio Source Separation, Springer, 2018. ⟨hal-01633858⟩

Share

Metrics

Record views

529

Files downloads

1012