Zero-shot cross-lingual transfer in instruction tuning of large language model

arXiv:2402.14778v1 Announce Type: new
Abstract: Instruction tuning (IT) is widely used to teach pretrained large language models (LLMs) to follow arbitrary instructions, but is under-studied in multilingual settings. In this work, we conduct a systematic study of zero-shot cross-lingual transfer in IT, when an LLM is instruction-tuned on English-only data and then tested on user prompts in other languages. We investigate the influence of model configuration choices and devise a multi-facet evaluation strategy for multilingual instruction following. We find that …

abstract arxiv cross-lingual cs.ai cs.cl data english instruction-tuned language language model language models large language large language model large language models llm llms multilingual study transfer type work zero-shot