Aksharantar: Towards building open transliteration tools for the next billion users. (arXiv:2205.03018v1 [cs.CL])

We introduce Aksharantar, the largest publicly available transliteration
dataset for 21 Indic languages containing 26 million transliteration pairs. We
build this dataset by mining transliteration pairs from large monolingual and
parallel corpora, as well as collecting transliterations from human annotators
to ensure diversity of words and representation of low-resource languages. We
introduce a new, large, diverse testset for Indic language transliteration
containing 103k words pairs spanning 19 languages that enables fine-grained
analysis of transliteration models.

We train the IndicXlit model …

arxiv building tools