Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

arXiv:2402.17840v1 Announce Type: cross
Abstract: Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation. We study the risk of datastore leakage in Retrieval-In-Context RAG Language Models (LMs). We show that an adversary can exploit LMs' instruction-following capabilities to easily extract text data verbatim from the datastore of RAG systems built with instruction-tuned LMs via prompt injection. The vulnerability exists for a wide range of modern LMs that span Llama2, Mistral/Mixtral, Vicuna, SOLAR, …

abstract arxiv capabilities context cs.ai cs.cl cs.cr cs.lg data data extraction exploit extraction knowledge language language models lms pre-trained models rag retrieval retrieval-augmented risk scalable show study systems test type