Take a Look at it! Rethinking How to Evaluate Language Model Jailbreak

arXiv:2404.06407v1 Announce Type: cross
Abstract: Large language models (LLMs) have become increasingly integrated with various applications. To ensure that LLMs do not generate unsafe responses, they are aligned with safeguards that specify what content is restricted. However, such alignment can be bypassed to produce prohibited content using a technique commonly referred to as jailbreak. Different systems have been proposed to perform the jailbreak automatically. These systems rely on evaluation methods to determine whether a jailbreak attempt is successful. However, our …

abstract alignment applications arxiv become cs.ai cs.cl cs.cr cs.lg generate however jailbreak language language model language models large language large language models llms look responses safeguards type