Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study

arXiv:2404.07060v1 Announce Type: cross
Abstract: We present an empirical study of groundedness in long-form question answering (LFQA) by retrieval-augmented large language models (LLMs). In particular, we evaluate whether every generated sentence is grounded in the retrieved documents or the model's pre-training data. Across 3 datasets and 4 model families, our findings reveal that a significant fraction of generated sentences are consistently ungrounded, even when those sentences contain correct ground-truth answers. Additionally, we examine the impacts of factors such as model …

abstract arxiv cs.cl cs.lg data datasets documents every families form generated language language models large language large language models llms pre-training question question answering retrieval retrieval-augmented study training training data type