Making Pre-trained Language Models Good Long-tailed Learners. (arXiv:2205.05461v1 [cs.CL])

Web: http://arxiv.org/abs/2205.05461

Prompt-tuning has shown appealing performance in few-shot classification by
virtue of its capability in effectively exploiting pre-trained knowledge. This
motivates us to check the hypothesis that prompt-tuning is also a promising
choice for long-tailed classification, since the tail classes are intuitively
few-shot ones. To achieve this aim, we conduct empirical studies to examine the
hypothesis. The results demonstrate that prompt-tuning exactly makes
pre-trained language models at least good long-tailed learners. For intuitions
on why prompt-tuning can achieve good performance in …

arxiv good language language models making models