Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments

arXiv:2403.13556v1 Announce Type: new
Abstract: In this work, we tackle the limitations of current LiDAR-based 3D object detection systems, which are hindered by a restricted class vocabulary and the high costs associated with annotating new object classes. Our exploration of open-vocabulary (OV) learning in urban environments aims to capture novel instances using pre-trained vision-language models (VLMs) with multi-sensor data. We design and benchmark a set of four potential solutions as baselines, categorizing them into either top-down or bottom-up approaches based …

3d object 3d object detection abstract arxiv class costs cs.ai cs.cv current detection environments exploration lidar limitations novel object systems type urban work