Defending Against Indirect Prompt Injection Attacks With Spotlighting

arXiv:2403.14720v1 Announce Type: cross
Abstract: Large Language Models (LLMs), while powerful, are built and trained to process a single text input. In common applications, multiple inputs can be processed by concatenating them together into a single stream of text. However, the LLM is unable to distinguish which sections of prompt belong to various input sources. Indirect prompt injection attacks take advantage of this vulnerability by embedding adversarial instructions into untrusted data being processed alongside user commands. Often, the LLM will …

abstract applications arxiv attacks cs.cl cs.cr cs.lg however inputs language language models large language large language models llm llms multiple process prompt prompt injection prompt injection attacks text them together type