Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks

arXiv:2305.14965v2 Announce Type: replace
Abstract: Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their …

abstract arxiv behavior breaches commercial cs.cl expert jailbreak language language models large language large language models llms privacy privacy and security prompts regulator security security breaches studies type