Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

arXiv:2404.14233v1 Announce Type: cross
Abstract: The rapidly developing Large Vision Language Models (LVLMs) have shown notable capabilities on a range of multi-modal tasks, but still face the hallucination phenomena where the generated texts do not align with the given contexts, significantly restricting the usages of LVLMs. Most previous work detects and mitigates hallucination at the coarse-grained level or requires expensive annotation (e.g., labeling by proprietary models or human experts). To address these issues, we propose detecting and mitigating hallucinations in …

abstract arxiv capabilities cs.ai cs.cl cs.cv cs.lg face feedback fine-grained generated hallucination language language models modal multi-modal tasks type via vision