Controllable Prosody Generation With Partial Inputs

arXiv:2303.09446v2 Announce Type: replace-cross
Abstract: We address the problem of human-in-the-loop control for generating prosody in the context of text-to-speech synthesis. Controlling prosody is challenging because existing generative models lack an efficient interface through which users can modify the output quickly and precisely. To solve this, we introduce a novel framework whereby the user provides partial inputs and the generative model generates the missing features. We propose a model that is specifically designed to encode partial prosodic features and output …

abstract arxiv context control cs.ai cs.cl cs.lg eess.as framework generative generative models human inputs loop novel solve speech synthesis text text-to-speech through type