Calibrated Language Models Must Hallucinate

arXiv:2311.14648v3 Announce Type: replace
Abstract: Recent language models generate false but plausible-sounding text with surprising frequency. Such "hallucinations" are an obstacle to the usability of language-based AI systems and can harm people who rely upon their outputs. This work shows that there is an inherent statistical lower-bound on the rate that pretrained language models hallucinate certain types of facts, having nothing to do with the transformer LM architecture or data quality. For "arbitrary" facts whose veracity cannot be determined from …

abstract ai systems arxiv cs.ai cs.cl false generate hallucinations harm language language models people rate shows statistical systems text type usability work