Exploring the Distinctiveness and Fidelity of the Descriptions Generated by Large Vision-Language Models

arXiv:2404.17534v1 Announce Type: new
Abstract: Large Vision-Language Models (LVLMs) are gaining traction for their remarkable ability to process and integrate visual and textual data. Despite their popularity, the capacity of LVLMs to generate precise, fine-grained textual descriptions has not been fully explored. This study addresses this gap by focusing on \textit{distinctiveness} and \textit{fidelity}, assessing how models like Open-Flamingo, IDEFICS, and MiniGPT-4 can distinguish between similar objects and accurately describe visual features. We proposed the Textual Retrieval-Augmented Classification (TRAC) framework, which, …

arxiv cs.cv cs.mm fidelity generated language language models type vision vision-language vision-language models