Se concentrer sur les différences : une méthode d'évaluation subjective efficace pour la comparaison de systèmes de synthèse

Abstract : When trying to assess the effectiveness of a new speech synthesis method, researchers usually conduct subjective evaluations by randomly choosing a small set of samples, from the same domain, taken from a baseline system and the proposed one. When selecting them randomly, statistically, samples with almost no differences are evaluated and the global measure is smoothed which may lead to judge the improvement not significant. To solve this methodological flaw, we propose to compare speech synthesis systems on thousands of generated samples from various domains and to focus subjective evaluations on the most relevant ones by computing a normalized alignment cost between sample pairs. This process has been successfully applied both in the HTS statistical framework and in the unit selection approach. A comparison between tests involving most different samples and randomly chosen samples shows clearly that the proposed approach reveals significant differences between the systems.
Complete list of metadatas

https://hal.inria.fr/hal-01338918
Contributor : Damien Lolive <>
Submitted on : Wednesday, June 29, 2016 - 1:00:03 PM
Last modification on : Friday, January 11, 2019 - 3:15:16 PM

Identifiers

  • HAL Id : hal-01338918, version 1

Citation

Jonathan Chevelu, Damien Lolive, Sébastien Le Maguer, David Guennec. Se concentrer sur les différences : une méthode d'évaluation subjective efficace pour la comparaison de systèmes de synthèse. Journées d'Études sur la Parole, Jul 2016, Paris, France. ⟨hal-01338918⟩

Share

Metrics

Record views

618