Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs

arXiv:2404.09971v1 Announce Type: new
Abstract: Large language models (LLMs) are susceptible to hallucination, which sparked a widespread effort to detect and prevent them. Recent work attempts to mitigate hallucinations by intervening in the model's computation during generation, using different setups and heuristics. Those works lack separation between different hallucination causes. In this work, we first introduce an approach for constructing datasets based on the model knowledge for detection and intervention methods in closed-book and open-book question-answering settings. We then characterize …

arxiv benchmarks cs.cl hallucinations llms type