Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean

arXiv:2403.10882v1 Announce Type: new
Abstract: Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. Numerous big tech companies and research institutes have developed multilingual LLMs (MLLMs) to meet current demands, overlooking less-resourced languages (LRLs). This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs. First, the MLLM vocabularies of LRLs were expanded to enhance expressiveness. Second, bilingual data were used for pretraining to align …

abstract arxiv augmentation big big tech companies case case study companies computing computing resources cs.ai cs.cl current expansion however language language models languages large language large language models llms mllms multilingual pretraining research resources study tech type word