AraLegal-BERT: A pretrained language model for Arabic Legal text. (arXiv:2210.08284v1 [cs.CL])

The effectiveness of the BERT model on multiple linguistic tasks has been
well documented. On the other hand, its potentials for narrow and specific
domains such as Legal, have not been fully explored. In this paper, we examine
how BERT can be used in the Arabic legal domain and try customizing this
language model for several downstream tasks using several different
domain-relevant training and testing datasets to train BERT from scratch. We
introduce the AraLegal-BERT, a bidirectional encoder Transformer-based model …

arxiv bert language language model legal pretrained language model text