Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations. (arXiv:2210.16637v2 [cs.CL] UPDATED)

Recent work has demonstrated that pre-trained language models (PLMs) are
zero-shot learners. However, most existing zero-shot methods involve heavy
human engineering or complicated self-training pipelines, hindering their
application to new situations. In this work, we show that zero-shot text
classification can be improved simply by clustering texts in the embedding
spaces of PLMs. Specifically, we fit the unlabeled texts with a Bayesian
Gaussian Mixture Model after initializing cluster positions and shapes using
class names. Despite its simplicity, this approach achieves …

arxiv clustering language language models making