When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively

arXiv:2404.19705v1 Announce Type: new
Abstract: In this paper, we demonstrate how Large Language Models (LLMs) can effectively learn to use an off-the-shelf information retrieval (IR) system specifically when additional context is required to answer a given question. Given the performance of IR systems, the optimal strategy for question answering does not always entail external information retrieval; rather, it often involves leveraging the parametric memory of the LLM itself. Prior research has identified this phenomenon in the PopQA dataset, wherein the …

abstract arxiv context cs.cl cs.ir information language language models large language large language models learn llms paper performance question question answering retrieval strategy systems teaching type