Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Improving Sound Event Detection Metrics: Insights from DCASE 2020

Giacomo Ferroni 1 Nicolas Turpault 2 Juan Azcarreta 1 Francesco Tuveri 1 Romain Serizel 2 Çagdaş Bilen 1 Sacha Krstulović 1
2 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : The ranking of sound event detection (SED) systems may be biased by assumptions inherent to evaluation criteria and to the choice of an operating point. This paper compares conventional event-based and segment-based criteria against the Polyphonic Sound Detection Score (PSDS)'s intersection-based criterion, over a selection of systems from DCASE 2020 Challenge Task 4. It shows that, by relying on collars , the conventional event-based criterion introduces different strictness levels depending on the length of the sound events, and that the segment-based criterion may lack precision and be application dependent. Alternatively, PSDS's intersection-based criterion overcomes the dependency of the evaluation on sound event duration and provides robustness to labelling subjectivity, by allowing valid detections of interrupted events. Furthermore, PSDS enhances the comparison of SED systems by measuring sound event modelling performance independently from the systems' operating points.
Complete list of metadatas

Cited literature [20 references]  Display  Hide  Download

https://hal.inria.fr/hal-02978422
Contributor : Romain Serizel <>
Submitted on : Monday, October 26, 2020 - 4:05:32 PM
Last modification on : Tuesday, October 27, 2020 - 10:54:28 AM

Files

ICASSP2021_task4_PSDS.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02978422, version 1
  • ARXIV : 2010.13648

Collections

Citation

Giacomo Ferroni, Nicolas Turpault, Juan Azcarreta, Francesco Tuveri, Romain Serizel, et al.. Improving Sound Event Detection Metrics: Insights from DCASE 2020. 2020. ⟨hal-02978422⟩

Share

Metrics

Record views

72

Files downloads

94