Is the System Message Really Important to Jailbreaks in Large Language Models?

arXiv:2402.14857v1 Announce Type: new
Abstract: The rapid evolution of Large Language Models (LLMs) has rendered them indispensable in modern society. While security measures are typically in place to align LLMs with human values prior to release, recent studies have unveiled a concerning phenomenon named "jailbreak." This term refers to the unexpected and potentially harmful responses generated by LLMs when prompted with malicious questions. Existing research focuses on generating jailbreak prompts but our study aim to answer a different question: Is …

abstract arxiv cs.ai cs.cl cs.cr evolution human jailbreak language language models large language large language models llms modern prior release security society studies them type values