Learning New Tasks from a Few Examples with Soft-Label Prototypes

arXiv:2210.17437v3 Announce Type: replace
Abstract: Existing approaches to few-shot learning in NLP rely on large language models and fine-tuning of these to generalise on out-of-distribution data. In this work, we propose a simple yet powerful approach to "extreme" few-shot learning, wherein models are exposed to as little as 4 examples per class, based on soft-label prototypes that collectively capture the distribution of different classes across the input domain space. Inspired by previous work (Sucholutsky et al., 2021) on univariate or …

abstract arxiv cs.cl cs.lg data distribution examples few-shot few-shot learning fine-tuning language language models large language large language models nlp simple tasks type work