CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations

arXiv:2404.06690v1 Announce Type: cross
Abstract: Recent advancements in zero-shot text-to-speech (TTS) modeling have led to significant strides in generating high-fidelity and diverse speech. However, dialogue generation, along with achieving human-like naturalness in speech, continues to be a challenge in the field. In this paper, we introduce CoVoMix: Conversational Voice Mixture Generation, a novel model for zero-shot, human-like, multi-speaker, multi-round dialogue speech generation. CoVoMix is capable of first converting dialogue text into multiple streams of discrete tokens, with each token stream …

arxiv conversations cs.ai cs.cl cs.lg cs.sd eess.as human human-like speech speech generation type zero-shot