Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling. (arXiv:2111.12698v2 [cs.CV] UPDATED)

Open-vocabulary instance segmentation aims at segmenting novel classes
without mask annotations. It is an important step toward reducing laborious
human supervision. Most existing works first pretrain a model on captioned
images covering many novel classes and then finetune it on limited base classes
with mask annotations. However, the high-level textual information learned from
caption pretraining alone cannot effectively encode the details required for
pixel-wise segmentation. To address this, we propose a cross-modal
pseudo-labeling framework, which generates training pseudo masks by …

arxiv cv labeling segmentation