Are Vision Language Models Texture or Shape Biased and Can We Steer Them?

arXiv:2403.09193v1 Announce Type: cross
Abstract: Vision language models (VLMs) have drastically changed the computer vision model landscape in only a few years, opening an exciting array of new applications from zero-shot image classification, over to image captioning, and visual question answering. Unlike pure vision models, they offer an intuitive way to access visual content through language prompting. The wide applicability of such models encourages us to ask whether they also align with human vision - specifically, how far they adopt …

abstract applications array arxiv captioning classification computer computer vision cs.ai cs.cv cs.lg image landscape language language models q-bio.nc question question answering texture them type vision vision models visual vlms zero-shot