Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription

arXiv:2403.10953v1 Announce Type: new
Abstract: Large image diffusion models have demonstrated zero-shot capability in novel view synthesis (NVS). However, existing diffusion-based NVS methods struggle to generate novel views that are accurately consistent with the corresponding ground truth poses and appearances, even on the training set. This consequently limits the performance of downstream tasks, such as image-to-multiview generation and 3D reconstruction. We realize that such inconsistency is largely due to the fact that it is difficult to enforce accurate pose and …

abstract arxiv capability consistent cs.cv diffusion diffusion models generate however image image diffusion loop novel performance set struggle synthesis training transcription truth type via view zero-shot