Vocabulary-free Image Classification and Semantic Segmentation

arXiv:2404.10864v1 Announce Type: new
Abstract: Large vision-language models revolutionized image classification and semantic segmentation paradigms. However, they typically assume a pre-defined set of categories, or vocabulary, at test time for composing textual prompts. This assumption is impractical in scenarios with unknown or evolving semantic context. Here, we address this issue and introduce the Vocabulary-free Image Classification (VIC) task, which aims to assign a class from an unconstrained language-induced semantic space to an input image without needing a known vocabulary. VIC …

abstract arxiv classification context cs.cv free however image issue language language models prompts segmentation semantic set test textual type vision vision-language vision-language models