Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps

arXiv:2307.05052v3 Announce Type: replace
Abstract: We investigate the role of various demonstration components in the in-context learning (ICL) performance of large language models (LLMs). Specifically, we explore the impacts of ground-truth labels, input distribution, and complementary explanations, particularly when these are altered or perturbed. We build on previous work, which offers mixed findings on how these elements influence ICL. To probe these questions, we employ explainable NLP (XNLP) methods and utilize saliency maps of contrastive demonstrations for both qualitative and …

arxiv context cs.ai cs.cl in-context learning maps type understanding