A Unified Approach for Text- and Image-guided 4D Scene Generation

arXiv:2311.16854v3 Announce Type: replace
Abstract: Large-scale diffusion generative models are greatly simplifying image, video and 3D asset creation from user-provided text prompts and images. However, the challenging problem of text-to-4D dynamic 3D scene generation with diffusion guidance remains largely unexplored. We propose Dream-in-4D, which features a novel two-stage approach for text-to-4D synthesis, leveraging (1) 3D and 2D diffusion guidance to effectively learn a high-quality static 3D asset in the first stage; (2) a deformable neural radiance field that explicitly disentangles …

3d scene generation abstract arxiv cs.cv diffusion dynamic features generative generative models guidance however image images novel prompts scale simplifying stage text type video