No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement

arXiv:2404.15737v1 Announce Type: new
Abstract: Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language Models. However, a trade-off of this approach is the reduction in positive transfer learning from closely related languages. In response, we introduce a novel method called language arithmetic, which enables training-free post-processing to address this limitation. Inspired by the task arithmetic framework, we apply learning via addition to …

arxiv cs.cl free language train training type