Feature-Adaptive and Data-Scalable In-Context Learning

arXiv:2405.10738v1 Announce Type: new
Abstract: In-context learning (ICL), which promotes inference with several demonstrations, has become a widespread paradigm to stimulate LLM capabilities for downstream tasks. Due to context length constraints, it cannot be further improved in spite of more training data, and general features directly from LLMs in ICL are not adaptive to the specific downstream task. In this paper, we propose a feature-adaptive and data-scalable in-context learning framework (FADS-ICL), which can leverage task-adaptive features to promote inference on …

arxiv context context learning cs.cl data feature in-context learning scalable type