Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking

arXiv:2404.06742v1 Announce Type: new
Abstract: Detecting non-factual content is a longstanding goal to increase the trustworthiness of large language models (LLMs) generations. Current factuality probes, trained using humanannotated labels, exhibit limited transferability to out-of-distribution content, while online selfconsistency checking imposes extensive computation burden due to the necessity of generating multiple outputs. This paper proposes PINOSE, which trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data and achieving transferability across diverse data distributions. As …

abstract arxiv computation cs.cl current detection distribution labels language language models large language large language models llms offline probe training type via