Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindi

arXiv:2309.10272v2 Announce Type: replace
Abstract: One of the most popular downstream tasks in the field of Natural Language Processing is text classification. Text classification tasks have become more daunting when the texts are code-mixed. Though they are not exposed to such text during pre-training, different BERT models have demonstrated success in tackling Code-Mixed NLP challenges. Again, in order to enhance their performance, Code-Mixed NLP models have depended on combining synthetic data with real-world data. It is crucial to understand how …

abstract arxiv become bert bert models classification code cs.cl english hindi language language processing mixed modeling natural natural language natural language processing popular pre-training processing success tasks text text classification training type