Generalizable Semantic Vision Query Generation for Zero-shot Panoptic and Semantic Segmentation

arXiv:2402.13697v1 Announce Type: new
Abstract: Zero-shot Panoptic Segmentation (ZPS) aims to recognize foreground instances and background stuff without images containing unseen categories in training. Due to the visual data sparsity and the difficulty of generalizing from seen to unseen categories, this task remains challenging. To better generalize to unseen classes, we propose Conditional tOken aligNment and Cycle trAnsiTion (CONCAT), to produce generalizable semantic vision queries. First, a feature extractor is trained by CON to link the vision and semantics for …

abstract arxiv cs.cv data images instances panoptic segmentation query segmentation semantic sparsity training type vision visual visual data zero-shot