NeRFFaceSpeech: One-shot Audio-diven 3D Talking Head Synthesis via Generative Prior

arXiv:2405.05749v1 Announce Type: new
Abstract: Audio-driven talking head generation is advancing from 2D to 3D content. Notably, Neural Radiance Field (NeRF) is in the spotlight as a means to synthesize high-quality 3D talking head outputs. Unfortunately, this NeRF-based approach typically requires a large number of paired audio-visual data for each identity, thereby limiting the scalability of the method. Although there have been attempts to generate audio-driven 3D talking head animations with a single image, the results are often unsatisfactory due …

abstract arxiv audio cs.cv data generative head in the spotlight nerf neural radiance field prior quality spotlight synthesis talking head type via visual visual data