Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision

arXiv:2402.16508v1 Announce Type: new
Abstract: Cross-lingual question answering (CLQA) is a complex problem, comprising cross-lingual retrieval from a multilingual knowledge base, followed by answer generation either in English or the query language. Both steps are usually tackled by separate models, requiring substantial annotated datasets, and typically auxiliary resources, like machine translation systems to bridge between languages. In this paper, we show that CLQA can be addressed using a single encoder-decoder model. To effectively train this model, we propose a self-supervised …

abstract arxiv cross-lingual cs.cl cs.ir datasets domain english knowledge knowledge base language machine multilingual pre-training query query language question question answering resources retrieval scale supervision synthetic training type