Calibrating the Confidence of Large Language Models by Eliciting Fidelity

arXiv:2404.02655v1 Announce Type: new
Abstract: Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless. However, post-alignment, these language models often exhibit overconfidence, where the expressed confidence does not accurately calibrate with their correctness rate. In this paper, we decompose the language model confidence into the \textit{Uncertainty} about the question and the \textit{Fidelity} to the answer generated by language models. Then, we propose a plug-and-play method to estimate the confidence of language models. …

abstract alignment arxiv confidence cs.cl fidelity good however language language model language models large language large language models paper rate rlhf type