Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

arXiv:2310.06387v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have shown remarkable success in various tasks, but concerns about their safety and the potential for generating harmful content have emerged. In this paper, we delve into the potential of In-Context Learning (ICL) to modulate the alignment of LLMs. Specifically, we propose the In-Context Attack (ICA), which employs strategically crafted harmful demonstrations to subvert LLMs, and the In-Context Defense (ICD), which bolsters model resilience through examples that demonstrate refusal to produce …

abstract alignment arxiv concerns context context learning cs.ai cs.cl cs.cr cs.lg in-context learning jailbreak language language models large language large language models llms modulate paper safety success tasks type