VLSM-Adapter: Finetuning Vision-Language Segmentation Efficiently with Lightweight Blocks

arXiv:2405.06196v1 Announce Type: cross
Abstract: Foundation Vision-Language Models (VLMs) trained using large-scale open-domain images and text pairs have recently been adapted to develop Vision-Language Segmentation Models (VLSMs) that allow providing text prompts during inference to guide image segmentation. If robust and powerful VLSMs can be built for medical images, it could aid medical professionals in many clinical tasks where they must spend substantial time delineating the target structure of interest. VLSMs for medical images resort to fine-tuning base VLM or …

adapter arxiv cs.ai cs.cl cs.cv cs.lg finetuning language segmentation type vision vision-language