SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

arXiv:2311.08370v2 Announce Type: replace
Abstract: The past year has seen rapid acceleration in the development of large language models (LLMs). However, without proper steering and safeguards, LLMs will readily follow malicious instructions, provide unsafe advice, and generate toxic content. We introduce SimpleSafetyTests (SST) as a new test suite for rapidly and systematically identifying such critical safety risks. The test suite comprises 100 test prompts across five harm areas that LLMs, for the vast majority of applications, should refuse to comply …

abstract advice arxiv cs.cl development generate language language models large language large language models llms risks safeguards safety simplesafetytests test type will