DIG3D: Marrying Gaussian Splatting with Deformable Transformer for Single Image 3D Reconstruction

arXiv:2404.16323v1 Announce Type: new
Abstract: In this paper, we study the problem of 3D reconstruction from a single-view RGB image and propose a novel approach called DIG3D for 3D object reconstruction and novel view synthesis. Our method utilizes an encoder-decoder framework which generates 3D Gaussians in decoder with the guidance of depth-aware image features from encoder. In particular, we introduce the use of deformable transformer, allowing efficient and effective decoding through 3D reference point and multi-layer refinement adaptations. By harnessing …

3d object 3d reconstruction abstract arxiv cs.cv decoder encoder encoder-decoder framework image novel object paper study synthesis transformer type view