NICE: To Optimize In-Context Examples or Not?

Recent works have shown that large language models (LLMs) work remarkably well on a wide range of tasks through in-context learning and optimization of in-context examples (ICE). However, most of these studies assume either a fixed or no instruction provided in the prompt, leading to the apparent consensus that the optimization of in-context examples is critical for better performance. We challenge this consensus for instruction-tuned LLMs by investigating the necessity of optimizing in-context examples when task-specific instructions are provided, and …

consensus context cs.ai cs.cl cs.lg examples ice in-context learning language language models large language large language models llms nice optimization prompt studies tasks the prompt through work