Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion

arXiv:2312.04466v2 Announce Type: replace
Abstract: Existing methods for synthesizing 3D human gestures from speech have shown promising results, but they do not explicitly model the impact of emotions on the generated gestures. Instead, these methods directly output animations from speech without control over the expressed emotion. To address this limitation, we present AMUSE, an emotional speech-driven body animation model based on latent diffusion. Our observation is that content (i.e., gestures related to speech rhythm and word utterances), emotion, and personal …

abstract animation animations arxiv control cs.cv diffusion emotion emotions generated gestures human impact results speech type via