Skip to Main content Skip to Navigation
New interface
Conference papers

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

Abstract : We propose using self-supervised discrete representations for the task of speech resynthesis. To generate disentangled representation, we separately extract low-bitrate representations for speech content, prosodic information, and speaker identity. This allows to synthesize speech in a controllable manner. We analyze various state-of-the-art, self-supervised representation learning methods and shed light on the advantages of each method while considering reconstruction quality and disentanglement properties. Specifically, we evaluate the F0 reconstruction, speaker identification performance (for both resynthesis and voice conversion), recordings' intelligibility, and overall quality using subjective human evaluation. Lastly, we demonstrate how these representations can be used for an ultra-lightweight speech codec. Using the obtained representations, we can get to a rate of 365 bits per second while providing better speech quality than the baseline methods. Audio samples can be found under the following link: speechbot.github.io/resynthesis.
Complete list of metadata

https://hal.inria.fr/hal-03329245
Contributor : Emmanuel Dupoux Connect in order to contact the contributor
Submitted on : Monday, October 11, 2021 - 11:34:31 AM
Last modification on : Friday, November 18, 2022 - 9:23:15 AM
Long-term archiving on: : Wednesday, January 12, 2022 - 6:41:03 PM

File

2104.00355.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03329245, version 1
  • ARXIV : 2104.00355

Collections

Citation

Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, et al.. Speech Resynthesis from Discrete Disentangled Self-Supervised Representations. INTERSPEECH 2021 - Annual Conference of the International Speech Communication Association, Aug 2021, Brno, Czech Republic. ⟨hal-03329245⟩

Share

Metrics

Record views

52

Files downloads

152