SsciBERT: A Pre-trained Language Model for Social Science Texts. (arXiv:2206.04510v1 [cs.CL])

The academic literature of social sciences is the literature that records
human civilization and studies human social problems. With the large-scale
growth of this literature, ways to quickly find existing research on relevant
issues have become an urgent demand for researchers. Previous studies, such as
SciBERT, have shown that pre-training using domain-specific texts can improve
the performance of natural language processing tasks in those fields. However,
there is no pre-trained language model for social sciences, so this paper
proposes a …

arxiv language language model science social social science