Talking Face Generation with Multilingual TTS. (arXiv:2205.06421v1 [cs.CV])

In this work, we propose a joint system combining a talking face generation
system with a text-to-speech system that can generate multilingual talking face
videos from only the text input. Our system can synthesize natural multilingual
speeches while maintaining the vocal identity of the speaker, as well as lip
movements synchronized to the synthesized speech. We demonstrate the
generalization capabilities of our system by selecting four languages (Korean,
English, Japanese, and Chinese) each from a different language family. We also …

arxiv cv face generation tts