Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning

arXiv:2309.14771v2 Announce Type: replace
Abstract: Large language models (LLMs) enable in-context learning (ICL) by conditioning on a few labeled training examples as a text-based prompt, eliminating the need for parameter updates and achieving competitive performance. In this paper, we demonstrate that factual knowledge is imperative for the performance of ICL in three core facets: the inherent knowledge learned in LLMs, the factual knowledge derived from the selected in-context examples, and the knowledge biases in LLMs for output generation. To unleash …

abstract arxiv context cs.ai cs.cl examples in-context learning knowledge language language models large language large language models llms paper performance prompt text training type updates