LLM In-Context Recall is Prompt Dependent

arXiv:2404.08865v1 Announce Type: cross
Abstract: The proliferation of Large Language Models (LLMs) highlights the critical importance of conducting thorough evaluations to discern their comparative advantages, limitations, and optimal use cases. Particularly important is assessing their capacity to accurately retrieve information included in a given prompt. A model's ability to do this significantly influences how effectively it can utilize contextual details, thus impacting its practical efficacy and dependability in real-world applications.
Our research analyzes the in-context recall performance of various LLMs …

abstract advantages arxiv capacity cases context cs.cl cs.lg highlights importance information language language models large language large language models limitations llm llms prompt recall type use cases