Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Improving Sound Event Detection Metrics: Insights from DCASE 2020

Giacomo Ferroni 1 Nicolas Turpault 2 Juan Azcarreta 1 Francesco Tuveri 1 Romain Serizel 2 Çagdaş Bilen 1 Sacha Krstulović 1
2 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : The ranking of sound event detection (SED) systems may be biased by assumptions inherent to evaluation criteria and to the choice of an operating point. This paper compares conventional event-based and segment-based criteria against the Polyphonic Sound Detection Score (PSDS)'s intersection-based criterion, over a selection of systems from DCASE 2020 Challenge Task 4. It shows that, by relying on collars , the conventional event-based criterion introduces different strictness levels depending on the length of the sound events, and that the segment-based criterion may lack precision and be application dependent. Alternatively, PSDS's intersection-based criterion overcomes the dependency of the evaluation on sound event duration and provides robustness to labelling subjectivity, by allowing valid detections of interrupted events. Furthermore, PSDS enhances the comparison of SED systems by measuring sound event modelling performance independently from the systems' operating points.
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download
Contributor : Romain Serizel Connect in order to contact the contributor
Submitted on : Monday, October 26, 2020 - 4:05:32 PM
Last modification on : Wednesday, November 3, 2021 - 7:57:21 AM
Long-term archiving on: : Wednesday, January 27, 2021 - 6:53:19 PM


Files produced by the author(s)


  • HAL Id : hal-02978422, version 1
  • ARXIV : 2010.13648



Giacomo Ferroni, Nicolas Turpault, Juan Azcarreta, Francesco Tuveri, Romain Serizel, et al.. Improving Sound Event Detection Metrics: Insights from DCASE 2020. 2020. ⟨hal-02978422⟩



Record views


Files downloads