Image-Text Co-Decomposition for Text-Supervised Semantic Segmentation

arXiv:2404.04231v1 Announce Type: new
Abstract: This paper addresses text-supervised semantic segmentation, aiming to learn a model capable of segmenting arbitrary visual concepts within images by using only image-text pairs without dense annotations. Existing methods have demonstrated that contrastive learning on image-text pairs effectively aligns visual segments with the meanings of texts. We notice that there is a discrepancy between text alignment and semantic segmentation: A text often consists of multiple semantic concepts, whereas semantic segmentation strives to create semantically homogeneous …

abstract annotations arxiv concepts cs.cv image images learn paper segmentation semantic text type visual visual concepts