Exploring the Robustness of In-Context Learning with Noisy Labels

arXiv:2404.18191v1 Announce Type: cross
Abstract: Recently, the mysterious In-Context Learning (ICL) ability exhibited by Transformer architectures, especially in large language models (LLMs), has sparked significant research interest. However, the resilience of Transformers' in-context learning capabilities in the presence of noisy samples, prevalent in both training corpora and prompt demonstrations, remains underexplored. In this paper, inspired by prior research that studies ICL ability using simple function classes, we take a closer look at this problem by investigating the robustness of Transformers …

arxiv context cs.ai cs.cl cs.cr cs.lg in-context learning labels math.oc robustness type