ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models

arXiv:2403.01807v1 Announce Type: new
Abstract: 3D asset generation is getting massive amounts of attention, inspired by the recent success of text-guided 2D content creation. Existing text-to-3D methods use pretrained text-to-image diffusion models in an optimization problem or fine-tune them on synthetic data, which often results in non-photorealistic 3D objects without backgrounds. In this paper, we present a method that leverages pretrained text-to-image models as a prior, and learn to generate multi-view images in a single denoising process from real-world data. …

3d objects abstract arxiv attention consistent cs.cv data diffusion diffusion models image image diffusion image generation massive objects optimization photorealistic results success synthetic synthetic data text text-to-image them type