Image-to-Image Matching via Foundation Models: A New Perspective for Open-Vocabulary Semantic Segmentation

arXiv:2404.00262v1 Announce Type: new
Abstract: Open-vocabulary semantic segmentation (OVS) aims to segment images of arbitrary categories specified by class labels or captions. However, most previous best-performing methods, whether pixel grouping methods or region recognition methods, suffer from false matches between image features and category labels. We attribute this to the natural gap between the textual features and visual features. In this work, we rethink how to mitigate false matches from the perspective of image-to-image matching and propose a novel relation-aware …

abstract arxiv captions class cs.cv false features foundation however image images image-to-image labels perspective pixel recognition segment segmentation semantic type via