Online Embedding Multi-Scale CLIP Features into 3D Maps

arXiv:2403.18178v1 Announce Type: cross
Abstract: This study introduces a novel approach to online embedding of multi-scale CLIP (Contrastive Language-Image Pre-Training) features into 3D maps. By harnessing CLIP, this methodology surpasses the constraints of conventional vocabulary-limited methods and enables the incorporation of semantic information into the resultant maps. While recent approaches have explored the embedding of multi-modal features in maps, they often impose significant computational costs, lacking practicality for exploring unfamiliar environments in real time. Our approach tackles these challenges by …

3d maps abstract arxiv clip constraints cs.cv cs.ro embedding features image information language maps methodology novel pre-training scale semantic study training type