Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Despite advances in AI alignment, language models (LM) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries modify input prompts to induce harmful behavior. While some defenses have been proposed, they focus on narrow threat models and fall short of a strong defense, which we posit should be effective, universal, and practical. To achieve this, we propose the first adversarial objective for defending LMs against jailbreaking attacks and an algorithm, robust prompt optimization (RPO), that uses gradient-based token optimization …

advances adversarial adversarial attacks ai alignment alignment attacks behavior cs.ai cs.cl cs.cv cs.lg defense focus jailbreaking language language models narrow optimization posit prompt prompts robust threat vulnerable