Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models

arXiv:2311.07194v3 Announce Type: replace
Abstract: LLMs (Large Language Models) usually interact with users in the form of dialogue and generate responses following their instructions, which naturally require dialogue comprehension abilities. However, dialogue comprehension is a general language ability which is hard to be evaluated directly. In this work, we propose to perform the evaluation focusing on the factual consistency issue with the help of the dialogue summarization task. Besides evaluating and analyzing the dialogue summarization performance (DIAC-Sum) of different LLMs, …

abstract arxiv cs.cl dialogue form general generate however language language models large language large language models llms responses type work