In-Context Example Ordering Guided by Label Distributions

arXiv:2402.11447v1 Announce Type: new
Abstract: By allowing models to predict without task-specific training, in-context learning (ICL) with pretrained LLMs has enormous potential in NLP. However, a number of problems persist in ICL. In particular, its performance is sensitive to the choice and order of in-context examples. Given the same set of in-context examples with different orderings, model performance may vary between near random to near state-of-the-art. In this work, we formulate in-context example ordering as an optimization problem. We examine …

abstract arxiv context cs.cl example examples in-context learning llms nlp performance set task-specific training training type