SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs

arXiv:2404.00469v1 Announce Type: new
Abstract: We introduce a novel problem, i.e., the localization of an input image within a multi-modal reference map represented by a database of 3D scene graphs. These graphs comprise multiple modalities, including object-level point clouds, images, attributes, and relationships between objects, offering a lightweight and efficient alternative to conventional methods that rely on extensive image databases. Given the available modalities, the proposed method SceneGraphLoc learns a fixed-sized embedding for each node (i.e., representing an object instance) …

abstract arxiv cs.cv database graphs image images localization map modal multi-modal multiple novel object objects reference relationships type visual