ScanTalk: 3D Talking Heads from Unregistered Scans

arXiv:2403.10942v1 Announce Type: new
Abstract: Speech-driven 3D talking heads generation has emerged as a significant area of interest among researchers, presenting numerous challenges. Existing methods are constrained by animating faces with fixed topologies, wherein point-wise correspondence is established, and the number and order of points remains consistent across all identities the model can animate. In this work, we present ScanTalk, a novel framework capable of animating 3D faces in arbitrary topologies including scanned data. Our approach relies on the DiffusionNet …

abstract arxiv challenges consistent cs.cv presenting researchers scans speech type wise