Natural language guidance of high-fidelity text-to-speech with synthetic annotations

Text-to-speech models trained on large-scale datasets have demonstrated impressive in-context learning capabilities and naturalness. However, control of speaker identity and style in these models typically requires conditioning on reference speech recordings, limiting creative applications. Alternatively, natural language prompting of speaker identity and style has demonstrated promising results and provides an intuitive method of control. However, reliance on human-labeled descriptions prevents scaling to large datasets.
Our work bridges the gap between these two approaches. We propose a scalable method for labeling …

annotations applications capabilities context control creative cs.cl cs.sd datasets fidelity guidance identity in-context learning language natural natural language prompting reference scale speaker speech style synthetic text text-to-speech