RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding

arXiv:2304.00962v4 Announce Type: replace
Abstract: We propose a lightweight and scalable Regional Point-Language Contrastive learning framework, namely \textbf{RegionPLC}, for open-world 3D scene understanding, aiming to identify and recognize open-set objects and categories. Specifically, based on our empirical studies, we introduce a 3D-aware SFusion strategy that fuses 3D vision-language pairs derived from multiple 2D foundation models, yielding high-quality, dense region-level language descriptions without human 3D annotations. Subsequently, we devise a region-aware point-discriminative contrastive learning objective to enable robust and effective 3D …

arxiv cs.ai cs.cv language open-world regional type understanding world