Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance. (arXiv:2401.15687v2 [cs.CV] UPDATED)

The synthesis of 3D facial animations from speech has garnered considerable
attention. Due to the scarcity of high-quality 4D facial data and
well-annotated abundant multi-modality labels, previous methods often suffer
from limited realism and a lack of lexible conditioning. We address this
challenge through a trilogy. We first introduce Generalized Neural Parametric
Facial Asset (GNPFA), an efficient variational auto-encoder mapping facial
geometry and images to a highly generalized expression latent space, decoupling
expressions and identities. Then, we utilize GNPFA to …

animation animations arxiv attention challenge cs.cv data guidance labels quality speech synthesis through