Pre-training Transformers on Indian Legal Text. (arXiv:2209.06049v2 [cs.CL] UPDATED)

Natural Language Processing in the legal domain been benefited hugely by the
emergence of Transformer-based Pre-trained Language Models (PLMs) pre-trained
on legal text. There exist PLMs trained over European and US legal text, most
notably LegalBERT. However, with the rapidly increasing volume of NLP
applications on Indian legal documents, and the distinguishing characteristics
of Indian legal text, it has become necessary to pre-train LMs over Indian
legal text as well. In this work, we introduce transformer-based PLMs
pre-trained over a …

arxiv legal pre-training text training transformers