AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation

arXiv:2311.07397v2 Announce Type: replace-cross
Abstract: Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences. Therefore, evaluating MLLMs' hallucinations is becoming increasingly important in model improvement and practical application deployment. Previous works are limited in high evaluation costs (e.g., relying on humans or advanced LLMs) and insufficient evaluation dimensions (e.g., types of tasks and hallucinations). In this paper, we propose an LLM-free multi-dimensional benchmark AMBER, …

amber arxiv benchmark cs.cl cs.cv evaluation free hallucination llm mllms type