How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?

arXiv:2404.03302v1 Announce Type: new
Abstract: By leveraging the retrieval of information from external knowledge databases, Large Language Models (LLMs) exhibit enhanced capabilities for accomplishing many knowledge-intensive tasks. However, due to the inherent flaws of current retrieval systems, there might exist irrelevant information within those retrieving top-ranked passages. In this work, we present a comprehensive investigation into the robustness of LLMs to different types of irrelevant information under various conditions. We initially introduce a framework to construct high-quality irrelevant information that …

arxiv cs.cl inputs language language models large language large language models responses skew type