Auto-Vocabulary Semantic Segmentation

arXiv:2312.04539v2 Announce Type: replace
Abstract: Open-ended image understanding tasks gained significant attention from the research community, particularly with the emergence of Vision-Language Models. Open-Vocabulary Segmentation (OVS) methods are capable of performing semantic segmentation without relying on a fixed vocabulary, and in some cases, they operate without the need for training or fine-tuning. However, OVS methods typically require users to specify the vocabulary based on the task or dataset at hand. In this paper, we introduce \textit{Auto-Vocabulary Semantic Segmentation (AVS)}, advancing …

abstract arxiv attention auto cases community cs.cv emergence fine-tuning however image language language models research research community segmentation semantic tasks training type understanding vision vision-language models