Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models. (arXiv:2206.02246v2 [cs.SD] UPDATED)

We present a novel way of conditioning a pretrained denoising diffusion
speech model to produce speech in the voice of a novel person unseen during
training. The method requires a short (~3 seconds) sample from the target
person, and generation is steered at inference time, without any training
steps. At the heart of the method lies a sampling process that combines the
estimation of the denoising model with a low-pass version of the new speaker's
sample. The objective and subjective …

arxiv denoising diffusion tts voice