[R] EMNLP 2023: Fast and Accurate Factual Inconsistency Detection Over Long Documents IMPROVES Pre-Trained Model's Hallucination Detection Capabilities With No Fine-Tuning Necessary

TL;DR: Our new paper presents SCALE, a technique that improves hallucination detection capabilities of pre-trained models over short and long documents with no fine-tuning necessary by evaluating hypotheses against large chunks of text as opposed to the traditional sentence-by-sentence approach. Furthermore, we introduce ScreenEval — the most extensive dialogue-based dataset for factual inconsistency detection on long documents to date.

https://preview.redd.it/ihqok954ntwb1.png?width=1964&format=png&auto=webp&s=70dbce6f913b2909b7d170959077c1fe19791c42

Title: Fast and Accurate Factual Inconsistency Detection Over Long Documents

Installation: `pip install scale-score`

Paper: [https://arxiv.org/abs/2310.13189](https://arxiv.org/abs/2310.13189)

Code: [https://github.com/asappresearch/scale-score](https://github.com/asappresearch/scale-score)

Abstract: Generative …

capabilities detection documents emnlp fine-tuning hallucination machinelearning paper pre-trained models scale text