Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning

arXiv:2404.03658v1 Announce Type: new
Abstract: Recovering the 3D scene geometry from a single view is a fundamental yet ill-posed problem in computer vision. While classical depth estimation methods infer only a 2.5D scene representation limited to the image plane, recent approaches based on radiance fields reconstruct a full 3D representation. However, these methods still struggle with occluded regions since inferring geometry without visual observation requires (i) semantic knowledge of the surroundings, and (ii) reasoning about spatial context. We propose KYN, …

arxiv cs.cv improving language neighbors reasoning spatial type via view vision