LISA: Reasoning Segmentation via Large Language Model

arXiv:2308.00692v3 Announce Type: replace
Abstract: Although perception systems have made remarkable advancements in recent years, they still rely on explicit human instruction or pre-defined categories to identify the target objects before executing visual recognition tasks. Such systems cannot actively reason and comprehend implicit user intention. In this work, we propose a new segmentation task -- reasoning segmentation. The task is designed to output a segmentation mask given a complex and implicit query text. Furthermore, we establish a benchmark comprising over …

arxiv cs.cv language language model large language large language model reasoning segmentation type via