Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation. (arXiv:2201.07786v1 [cs.GR])

Animating high-fidelity video portrait with speech audio is crucial for
virtual reality and digital entertainment. While most previous studies rely on
accurate explicit structural information, recent works explore the implicit
scene representation of Neural Radiance Fields (NeRF) for realistic generation.
In order to capture the inconsistent motions as well as the semantic difference
between human head and torso, some work models them via two individual sets of
NeRF, leading to unnatural results. In this work, we propose Semantic-aware
Speaking Portrait …

arxiv audio semantic video