Don't Say No: Jailbreaking LLM by Suppressing Refusal

arXiv:2404.16369v1 Announce Type: new
Abstract: Ensuring the safety alignment of Large Language Models (LLMs) is crucial to generating responses consistent with human values. Despite their ability to recognize and avoid harmful queries, LLMs are vulnerable to "jailbreaking" attacks, where carefully crafted prompts elicit them to produce toxic content. One category of jailbreak attacks is reformulating the task as adversarial attacks by eliciting the LLM to generate an affirmative response. However, the typical attack in this category GCG has very limited …

abstract alignment arxiv attacks consistent cs.cl human jailbreak jailbreaking language language models large language large language models llm llms prompts queries responses safety them type values vulnerable