Listen, Disentangle, and Control: Controllable Speech-Driven Talking Head Generation

arXiv:2405.07257v1 Announce Type: new
Abstract: Most earlier investigations on talking face generation have focused on the synchronization of lip motion and speech content. However, human head pose and facial emotions are equally important characteristics of natural human faces. While audio-driven talking face generation has seen notable advancements, existing methods either overlook facial emotions or are limited to specific individuals and cannot be applied to arbitrary subjects. In this paper, we propose a one-shot Talking Head Generation framework (SPEAK) that distinguishes …

arxiv control cs.cv head speech talking head type