Training a Bilingual Language Model by Mapping Tokens onto a Shared Character Space

arXiv:2402.16065v1 Announce Type: cross
Abstract: We train a bilingual Arabic-Hebrew language model using a transliterated version of Arabic texts in Hebrew, to ensure both languages are represented in the same script. Given the morphological, structural similarities, and the extensive number of cognates shared among Arabic and Hebrew, we assess the performance of a language model that employs a unified script for both languages, on machine translation which requires cross-lingual knowledge. The results are promising: our model outperforms a contrasting model …

abstract arabic arxiv bilingual cs.cl cs.lg language language model languages mapping space tokens train training type