Empirical study of pretrained multilingual language models for zero-shot cross-lingual knowledge transfer in generation

arXiv:2310.09917v3 Announce Type: replace
Abstract: Zero-shot cross-lingual knowledge transfer enables the multilingual pretrained language model (mPLM), finetuned on a task in one language, make predictions for this task in other languages. While being broadly studied for natural language understanding tasks, the described setting is understudied for generation. Previous works notice a frequent problem of generation in a wrong language and propose approaches to address it, usually using mT5 as a backbone model. In this work, we test alternative mPLMs, such …

abstract arxiv cross-lingual cs.cl knowledge language language model language models languages language understanding multilingual natural natural language predictions pretrained language model study tasks transfer type understanding zero-shot