Explore In-Context Segmentation via Latent Diffusion Models

arXiv:2403.09616v1 Announce Type: new
Abstract: In-context segmentation has drawn more attention with the introduction of vision foundation models. Most existing approaches adopt metric learning or masked image modeling to build the correlation between visual prompts and input image queries. In this work, we explore this problem from a new perspective, using one representative generation model, the latent diffusion model (LDM). We observe a task gap between generation and segmentation in diffusion models, but LDM is still an effective minimalist for …

abstract arxiv attention build context correlation cs.cv diffusion diffusion models explore foundation image introduction latent diffusion models modeling perspective prompts queries segmentation type via vision visual work