Teaching Large Language Models an Unseen Language on the Fly

arXiv:2402.19167v1 Announce Type: new
Abstract: Existing large language models struggle to support numerous low-resource languages, particularly the extremely low-resource ones where there is minimal training data available for effective parameter updating. We thus investigate whether LLMs can learn a new language on the fly solely through prompting. To study this question, we collect a research suite for Zhuang, a language supported by no LLMs currently. We introduce \textsc{DiPMT++}, a framework for adapting LLMs to unseen languages by in-context learning. Using …

abstract arxiv cs.cl data fly language language models languages large language large language models learn llms low prompting struggle study support teaching through training training data type