Investigating the prompt leakage effect and black-box defenses for multi-turn LLM interactions

arXiv:2404.16251v1 Announce Type: cross
Abstract: Prompt leakage in large language models (LLMs) poses a significant security and privacy threat, particularly in retrieval-augmented generation (RAG) systems. However, leakage in multi-turn LLM interactions along with mitigation strategies has not been studied in a standardized manner. This paper investigates LLM vulnerabilities against prompt leakage across 4 diverse domains and 10 closed- and open-source LLMs. Our unique multi-turn threat model leverages the LLM's sycophancy effect and our analysis dissects task instruction and knowledge leakage …

abstract arxiv box cs.ai cs.cl cs.cr however interactions language language models large language large language models llm llms paper privacy prompt rag retrieval retrieval-augmented security security and privacy strategies systems the prompt threat type vulnerabilities