The More You See in 2D, the More You Perceive in 3D

arXiv:2404.03652v1 Announce Type: new
Abstract: Humans can infer 3D structure from 2D images of an object based on past experience and improve their 3D understanding as they see more images. Inspired by this behavior, we introduce SAP3D, a system for 3D reconstruction and novel view synthesis from an arbitrary number of unposed images. Given a few unposed images of an object, we adapt a pre-trained view-conditioned diffusion model together with the camera poses of the images via test-time fine-tuning. The …

3d reconstruction abstract arxiv behavior cs.cv experience humans images novel object synthesis type understanding view