Generative Spoken Dialogue Language Modeling: preprint version - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

Generative Spoken Dialogue Language Modeling: preprint version

Résumé

We introduce dGSLM, the first "textless" model able to generate audio samples of naturalistic spoken dialogues. It uses recent work on unsupervised spoken unit discovery coupled with a dual-tower transformer architecture with cross-attention trained on 2000 hours of two-channel raw conversational audio (Fisher dataset) without any text or labels. It is able to generate speech, laughter and other paralinguistic signals in the two channels simultaneously and reproduces naturalistic turn taking. Generation samples can be found at: https://speechbot.github.io/dgslm.

Dates et versions

hal-03834730 , version 1 (30-10-2022)

Identifiants

Citer

Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, et al.. Generative Spoken Dialogue Language Modeling: preprint version. 2022. ⟨hal-03834730⟩
63 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More