CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks

arXiv:2402.16767v1 Announce Type: cross
Abstract: Knowledge-intensive language tasks (KILTs) typically require retrieving relevant documents from trustworthy corpora, e.g., Wikipedia, to produce specific answers. Very recently, a pre-trained generative retrieval model for KILTs, named CorpusBrain, was proposed and reached new state-of-the-art retrieval performance. However, most existing research on KILTs, including CorpusBrain, has predominantly focused on a static document collection, overlooking the dynamic nature of real-world scenarios, where new documents are continuously being incorporated into the source corpus. To address this gap, …

abstract art arxiv continual cs.cl cs.ir documents framework generative generative retrieval knowledge language performance pre-training research retrieval state tasks training trustworthy type wikipedia