GSVA: Generalized Segmentation via Multimodal Large Language Models

arXiv:2312.10103v2 Announce Type: replace
Abstract: Generalized Referring Expression Segmentation (GRES) extends the scope of classic RES to refer to multiple objects in one expression or identify the empty targets absent in the image. GRES poses challenges in modeling the complex spatial relationships of the instances in the image and identifying non-existing referents. Multimodal Large Language Models (MLLMs) have recently shown tremendous progress in these complicated vision-language tasks. Connecting Large Language Models (LLMs) and vision models, MLLMs are proficient in understanding …

abstract arxiv challenges cs.cv generalized identify image instances language language models large language large language models modeling multimodal multiple objects relationships segmentation spatial targets type via