Fine-grained Hallucination Detection and Editing for Language Models

arXiv:2401.06855v3 Announce Type: replace
Abstract: Large language models (LMs) are prone to generate factual errors, which are often called hallucinations. In this paper, we introduce a comprehensive taxonomy of hallucinations and argue that hallucinations manifest in diverse forms, each requiring varying degrees of careful assessments to verify factuality. We propose a novel task of automatic fine-grained hallucination detection and construct a new evaluation benchmark, FavaBench, that includes about one thousand fine-grained human judgments on three LM outputs across various domains. …

abstract arxiv cs.cl detection diverse editing errors fine-grained forms generate hallucination hallucinations language language models large language large language models lms manifest novel paper taxonomy type verify