Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations. (arXiv:2202.04582v1 [cs.CL])

Topic models have been the prominent tools for automatic topic discovery from
text corpora. Despite their effectiveness, topic models suffer from several
limitations including the inability of modeling word ordering information in
documents, the difficulty of incorporating external linguistic knowledge, and
the lack of both accurate and efficient inference methods for approximating the
intractable posterior. Recently, pretrained language models (PLMs) have brought
astonishing performance improvements to a wide variety of tasks due to their
superior representations of text. Interestingly, there …

arxiv clustering discovery language language model pretrained language model space