Evaluation of Audio Source Separation Models Using Hypothesis-Driven Non-Parametric Statistical Methods

Andrew Simpson 1 Gerard Roma 1 Emad Grais 1 Russell Mason 1 Chris Hummersone 1 Antoine Liutkus 2 Mark Plumbley 1
2 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : Audio source separation models are typically evaluated using objective separation quality measures, but rigorous statistical methods have yet to be applied to the problem of model comparison. As a result, it can be difficult to establish whether or not reliable progress is being made during the development of new models. In this paper, we provide a hypothesis-driven statistical analysis of the results of the recent source separation SiSEC challenge involving twelve competing models tested on separation of voice and accompaniment from fifty pieces of " professionally produced " contemporary music. Using non-parametric statistics, we establish reliable evidence for meaningful conclusions about the performance of the various models.
Document type :
Conference papers
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.inria.fr/hal-01410176
Contributor : Antoine Liutkus <>
Submitted on : Tuesday, December 6, 2016 - 2:52:56 PM
Last modification on : Sunday, November 17, 2019 - 1:12:05 AM
Long-term archiving on: Tuesday, March 21, 2017 - 5:49:43 AM

File

Separation_Stats.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01410176, version 1

Collections

Citation

Andrew Simpson, Gerard Roma, Emad Grais, Russell Mason, Chris Hummersone, et al.. Evaluation of Audio Source Separation Models Using Hypothesis-Driven Non-Parametric Statistical Methods. European Signal Processing Conference, EURASIP, Aug 2016, Budapest, Hungary. ⟨hal-01410176⟩

Share

Metrics

Record views

278

Files downloads

243