SpeechX: Neural Codec Language Model as a Versatile Speech Transformer

arXiv:2308.06873v2 Announce Type: replace-cross
Abstract: Recent advancements in generative speech models based on audio-text prompts have enabled remarkable innovations like high-quality zero-shot text-to-speech. However, existing models still face limitations in handling diverse audio-text speech generation tasks involving transforming input speech and processing audio captured in adverse acoustic conditions. This paper introduces SpeechX, a versatile speech generation model capable of zero-shot TTS and various speech transformation tasks, dealing with both clean and noisy signals. SpeechX combines neural codec language modeling with …

arxiv codec cs.cl cs.lg cs.sd eess.as language language model replace speech transformer type