Transcrib3D: 3D Referring Expression Resolution through Large Language Models

arXiv:2404.19221v1 Announce Type: new
Abstract: If robots are to work effectively alongside people, they must be able to interpret natural language references to objects in their 3D environment. Understanding 3D referring expressions is challenging -- it requires the ability to both parse the 3D structure of the scene and correctly ground free-form language in the presence of distraction and clutter. We introduce Transcrib3D, an approach that brings together 3D detection methods and the emergent reasoning capabilities of large language models …

arxiv cs.cl cs.cv language language models large language large language models resolution through type