Diffusion Models are Geometry Critics: Single Image 3D Editing Using Pre-Trained Diffusion Priors

arXiv:2403.11503v1 Announce Type: new
Abstract: We propose a novel image editing technique that enables 3D manipulations on single images, such as object rotation and translation. Existing 3D-aware image editing approaches typically rely on synthetic multi-view datasets for training specialized models, thus constraining their effectiveness on open-domain images featuring significantly more varied layouts and styles. In contrast, our method directly leverages powerful image diffusion models trained on a broad spectrum of text-image pairs and thus retain their exceptional generalization abilities. This …

abstract arxiv cs.cv datasets diffusion diffusion models domain editing geometry image images novel object rotation synthetic training translation type view