Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering

arXiv:2403.14197v1 Announce Type: new
Abstract: Retrieval-augmented generation models augment knowledge encoded in a language model by providing additional relevant external knowledge (context) during generation. Although it has been shown that the quantity and quality of context impact the performance of retrieval-augmented generation models during inference, limited research explores how these characteristics affect model training. This paper explores how context quantity and quality during model training affect the performance of Fusion-in-Decoder (FiD), the state-of-the-art retrieval-augmented generation model, in extractive open-domain question …

abstract arxiv context cs.cl decoder domain fusion impact inference knowledge language language model performance quality question question answering research retrieval retrieval-augmented training type