VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models

arXiv:2404.13874v1 Announce Type: cross
Abstract: Large Vision-Language Models (LVLMs) suffer from hallucination issues, wherein the models generate plausible-sounding but factually incorrect outputs, undermining their reliability. A comprehensive quantitative evaluation is necessary to identify and understand the extent of hallucinations in these models. However, existing benchmarks are often limited in scope, focusing mainly on object hallucinations. Furthermore, current evaluation methods struggle to effectively address the subtle semantic distinctions between model outputs and reference data, as well as the balance between hallucination …

abstract arxiv benchmarks coverage cs.cl cs.cv evaluation generate hallucination hallucinations however identify language language models quantitative reliability type vision vision-language vision-language models