Phylogeny-Inspired Adaptation of Multilingual Models to New Languages. (arXiv:2205.09634v2 [cs.CL] UPDATED)

Large pretrained multilingual models, trained on dozens of languages, have
delivered promising results due to cross-lingual learning capabilities on
variety of language tasks. Further adapting these models to specific languages,
especially ones unseen during pre-training, is an important goal towards
expanding the coverage of language technologies. In this study, we show how we
can use language phylogenetic information to improve cross-lingual transfer
leveraging closely related languages in a structured, linguistically-informed
manner. We perform adapter-based training on languages from diverse language …

arxiv