TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction

arXiv:2307.04642v2 Announce Type: replace
Abstract: When applied to open-domain question answering, large language models (LLMs) frequently generate incorrect responses based on made-up facts, which are called $\textit{hallucinations}$. Retrieval augmented generation (RAG) is a promising strategy to avoid hallucinations, but it does not provide guarantees on its correctness. To address this challenge, we propose the Trustworthy Retrieval Augmented Question Answering, or $\textit{TRAQ}$, which provides the first end-to-end statistical correctness guarantee for RAG. TRAQ uses conformal prediction, a statistical technique for constructing …

arxiv cs.ai cs.cl prediction question question answering retrieval trustworthy type via