CLIP Is Also a Good Teacher: A New Learning Framework for Inductive Zero-shot Semantic Segmentation

arXiv:2310.02296v2 Announce Type: replace
Abstract: Generalized Zero-shot Semantic Segmentation aims to segment both seen and unseen categories only under the supervision of the seen ones. To tackle this, existing methods adopt the large-scale Vision Language Models (VLMs) which obtain outstanding zero-shot performance. However, as the VLMs are designed for classification tasks, directly adapting the VLMs may lead to sub-optimal performance. Consequently, we propose CLIP-ZSS (Zero-shot Semantic Segmentation), a simple but effective training framework that enables any image encoder designed for …

abstract arxiv clip cs.cv framework generalized good inductive language language models performance scale segment segmentation semantic supervision type vision vlms zero-shot