Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training

arXiv:2403.02325v1 Announce Type: cross
Abstract: Highlighting particularly relevant regions of an image can improve the performance of vision-language models (VLMs) on various vision-language (VL) tasks by guiding the model to attend more closely to these regions of interest. For example, VLMs can be given a "visual prompt", where visual markers such as bounding boxes delineate key image regions. However, current VLMs that can incorporate visual guidance are either proprietary and expensive or require costly training on curated data that includes …

abstract arxiv cs.ai cs.cl cs.cv cs.lg example guidance highlighting image language language models performance prompt tasks training type vision vision-language models visual vlms