Cross-lingual Transfer of Monolingual Models. (arXiv:2109.07348v2 [cs.CL] UPDATED)

Recent studies in zero-shot cross-lingual learning using multilingual models
have falsified the previous hypothesis that shared vocabulary and joint
pre-training are the keys to cross-lingual generalization. Inspired by this
advancement, we introduce a cross-lingual transfer method for monolingual
models based on domain adaptation. We study the effects of such transfer from
four different languages to English. Our experimental results on GLUE show that
the transferred models outperform the native English model independently of the
source language. After probing the English …

arxiv cross-lingual transfer