Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance

arXiv:2312.10671v2 Announce Type: replace
Abstract: We introduce Open3DIS, a novel solution designed to tackle the problem of Open-Vocabulary Instance Segmentation within 3D scenes. Objects within 3D environments exhibit diverse shapes, scales, and colors, making precise instance-level identification a challenging task. Recent advancements in Open-Vocabulary scene understanding have made significant strides in this area by employing class-agnostic 3D instance proposal networks for object localization and learning queryable features for each 3D mask. While these methods produce high-quality instance proposals, they struggle …

3d scenes abstract arxiv colors cs.cv diverse environments guidance identification instance making novel objects segmentation solution type understanding