StableFace: Analyzing and Improving Motion Stability for Talking Face Generation. (arXiv:2208.13717v1 [cs.CV])

While previous speech-driven talking face generation methods have made
significant progress in improving the visual quality and lip-sync quality of
the synthesized videos, they pay less attention to lip motion jitters which
greatly undermine the realness of talking face videos. What causes motion
jitters, and how to mitigate the problem? In this paper, we conduct systematic
analyses on the motion jittering problem based on a state-of-the-art pipeline
that uses 3D face representations to bridge the input audio and output video, …

arxiv face generation