RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesis

arXiv:2404.03204v1 Announce Type: cross
Abstract: We present RALL-E, a robust language modeling method for text-to-speech (TTS) synthesis. While previous work based on large language models (LLMs) shows impressive performance on zero-shot TTS, such methods often suffer from poor robustness, such as unstable prosody (weird pitch and rhythm/duration) and a high word error rate (WER), due to the autoregressive prediction style of language models. The core idea behind RALL-E is chain-of-thought (CoT) prompting, which decomposes the task into simpler steps to …

abstract arxiv codec cs.ai cs.cl cs.lg cs.sd eess.as language language models large language large language models llms modeling performance pitch prompting robust robustness shows speech synthesis text text-to-speech thought tts type work zero-shot