ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models

arXiv:2311.09182v2 Announce Type: replace
Abstract: In recent times, large language models (LLMs) have shown impressive performance on various document-level tasks such as document classification, summarization, and question-answering. However, research on understanding their capabilities on the task of self-contradictions in long documents has been very limited. In this work, we introduce ContraDoc, the first human-annotated dataset to study self-contradictions in long documents across multiple domains, varying document lengths, self-contradictions types, and scope. We then analyze the current capabilities of four state-of-the-art …

arxiv cs.cl documents language language models large language large language models type understanding