Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks

arXiv:2402.12279v1 Announce Type: new
Abstract: Zero-shot cross-lingual generation implies finetuning of the multilingual pretrained language model on a generation task in one language and then using it to make predictions for this task in other languages. Previous works notice a frequent problem of generation in a wrong language and propose approaches to address it, usually using mT5 as a backbone model. In this work we compare various approaches proposed from the literature in unified settings, also including alternative backbone models, …

abstract arxiv cross-lingual cs.ai cs.cl finetuning generative key knowledge language language model languages multilingual predictions pretrained language model tasks transfer type zero-shot