Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

arXiv:2402.16315v1 Announce Type: new
Abstract: Recent advances in instruction-tuned Large Vision-Language Models (LVLMs) have imbued the models with the ability to generate high-level, image-grounded explanations with ease. While such capability is largely attributed to the rich world knowledge contained within the Large Language Models (LLMs), our work reveals their shortcomings in fine-grained visual categorization (FGVC) across six different benchmark settings. Most recent state-of-the-art LVLMs like LLaVa-1.5, InstructBLIP and GPT-4V not only severely deteriorate in terms of classification performance, e.g., average …

abstract advances arxiv capability concept cs.cl cs.cv fine-grained generate image instruction-tuned knowledge language language models large language large language models llms recognition type vision vision-language models visual work world