Unveiling Linguistic Regions in Large Language Models

arXiv:2402.14700v1 Announce Type: new
Abstract: Large Language Models (LLMs) have demonstrated considerable cross-lingual alignment and generalization ability. Current research primarily focuses on improving LLMs' cross-lingual generalization capabilities. However, there is still a lack of research on the intrinsic mechanisms of how LLMs achieve cross-lingual alignment. From the perspective of region partitioning, this paper conducts several investigations on the linguistic competence of LLMs. We discover a core region in LLMs that corresponds to linguistic competence, accounting for approximately 1% of the …

abstract alignment arxiv capabilities cross-lingual cs.cl current intrinsic language language models large language large language models llms paper partitioning perspective research type