WeLM: A Well-Read Pre-trained Language Model for Chinese. (arXiv:2209.10372v3 [cs.CL] UPDATED)

Large Language Models pre-trained with self-supervised learning have
demonstrated impressive zero-shot generalization capabilities on a wide
spectrum of tasks. In this work, we present WeLM: a well-read pre-trained
language model for Chinese that is able to seamlessly perform different types
of tasks with zero or few-shot demonstrations. WeLM is trained with 10B
parameters by "reading" a curated high-quality corpus covering a wide range of
topics. We show that WeLM is equipped with broad knowledge on various domains
and languages. On …

arxiv language language model