AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging

arXiv:2402.18913v1 Announce Type: new
Abstract: As an effective alternative to the direct fine-tuning on target tasks in specific languages, cross-lingual transfer addresses the challenges of limited training data by decoupling ''task ability'' and ''language ability'' by fine-tuning on the target task in the source language and another selected task in the target language, respectively. However, they fail to fully separate the task ability from the source language or the language ability from the chosen task. In this paper, we acknowledge …

abstract arxiv challenges cross-lingual cs.ai cs.cl data fine-tuning language language models languages large language large language models merging tasks training training data transfer type via