SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation

arXiv:2311.15537v2 Announce Type: replace
Abstract: Open-vocabulary semantic segmentation strives to distinguish pixels into different semantic groups from an open set of categories. Most existing methods explore utilizing pre-trained vision-language models, in which the key is to adopt the image-level model for pixel-level segmentation task. In this paper, we propose a simple encoder-decoder, named SED, for open-vocabulary semantic segmentation, which comprises a hierarchical encoder-based cost map generation and a gradual fusion decoder with category early rejection. The hierarchical encoder-based cost map …

arxiv cs.cv decoder encoder encoder-decoder segmentation semantic simple type