[Research] Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Paper: [https://arxiv.org/abs/2401.17263](https://arxiv.org/abs/2401.17263)


Abstract: Despite advances in AI alignment, language models (LM) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries modify input prompts to induce harmful behavior. While some defenses have been proposed, they focus on narrow threat models and fall short of a strong defense, which we posit should be effective, universal, and practical. To achieve this, we propose the first adversarial objective for defending LMs against jailbreaking attacks and an algorithm, robust prompt optimization (RPO), that uses …

abstract advances adversarial adversarial attacks ai alignment alignment attacks behavior defense focus jailbreaking language language models machinelearning narrow posit practical prompts threat vulnerable