all AI news
CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment
April 19, 2024, 4:47 a.m. | Geyu Lin, Bin Wang, Zhengyuan Liu, Nancy F. Chen
cs.CL updates on arXiv.org arxiv.org
Abstract: Multilingual proficiency presents a significant challenge for large language models (LLMs). English-centric models are usually suboptimal in other languages, particularly those that are linguistically distant from English. This performance discrepancy mainly stems from the imbalanced distribution of training data across languages during pre-training and instruction tuning stages. To address this problem, we propose a novel approach called CrossIn, which utilizes a mixed composition of cross-lingual instruction tuning data. Our method leverages the compressed representation shared …
abstract alignment arxiv challenge cross-lingual cs.ai cs.cl data distribution english knowledge language language models languages large language large language models llms multilingual performance pre-training training training data type
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Business Data Scientist, gTech Ads
@ Google | Mexico City, CDMX, Mexico
Lead, Data Analytics Operations
@ Zocdoc | Pune, Maharashtra, India