X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention

arXiv:2403.15931v1 Announce Type: new
Abstract: We propose X-Portrait, an innovative conditional diffusion model tailored for generating expressive and temporally coherent portrait animation. Specifically, given a single portrait as appearance reference, we aim to animate it with motion derived from a driving video, capturing both highly dynamic and subtle facial expressions along with wide-range head movements. As its core, we leverage the generative prior of a pre-trained diffusion model as the rendering backbone, while achieve fine-grained head pose and expression control …

abstract aim animation arxiv attention cs.ai cs.cv diffusion diffusion model driving dynamic hierarchical reference type video