HeRo: RoBERTa and Longformer Hebrew Language Models. (arXiv:2304.11077v1 [cs.CL])

In this paper, we fill in an existing gap in resources available to the
Hebrew NLP community by providing it with the largest so far pre-train dataset
HeDC4, a state-of-the-art pre-trained language model HeRo for standard length
inputs and an efficient transformer LongHeRo for long input sequences. The HeRo
model was evaluated on the sentiment analysis, the named entity recognition,
and the question answering tasks while the LongHeRo model was evaluated on the
document classification task with a dataset composed …

analysis art arxiv classification community dataset gap language language model language models nlp paper performance question answering resources roberta sentiment sentiment analysis standard state transformer