Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification

arXiv:2403.04696v1 Announce Type: cross
Abstract: Large language models (LLMs) are notorious for hallucinating, i.e., producing erroneous claims in their output. Such hallucinations can be dangerous, as occasional factual inaccuracies in the generated text might be obscured by the rest of the output being generally factual, making it extremely hard for the users to spot them. Current services that leverage LLMs usually do not provide any means for detecting unreliable generations. Here, we aim to bridge this gap. In particular, we …

abstract arxiv cs.ai cs.cl cs.lg fact-checking generated hallucinations language language models large language large language models llms making quantification rest text token type uncertainty via