Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models

arXiv:2308.07706v3 Announce Type: replace-cross
Abstract: Medical image segmentation allows quantifying target structure size and shape, aiding in disease diagnosis, prognosis, surgery planning, and comprehension.Building upon recent advancements in foundation Vision-Language Models (VLMs) from natural image-text pairs, several studies have proposed adapting them to Vision-Language Segmentation Models (VLSMs) that allow using language text as an additional input to segmentation models. Introducing auxiliary information via text with human-in-the-loop prompting during inference opens up unique opportunities, such as open vocabulary segmentation and potentially …

arxiv cs.ai cs.cl cs.cv cs.lg image language language models medical replace segmentation transfer transfer learning type vision vision-language vision-language models