ClimateBert: A Pretrained Language Model for Climate-Related Text. (arXiv:2110.12010v2 [cs.CL] UPDATED)

Over the recent years, large pretrained language models (LM) have
revolutionized the field of natural language processing (NLP). However, while
pretraining on general language has been shown to work very well for common
language, it has been observed that niche language poses problems. In
particular, climate-related texts include specific language that common LMs can
not represent accurately. We argue that this shortcoming of today's LMs limits
the applicability of modern NLP to the broad field of text processing of
climate-related …

arxiv climate language language model pretrained language model text