ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

arXiv:2404.08676v1 Announce Type: cross
Abstract: When building Large Language Models (LLMs), it is paramount to bear safety in mind and protect them with guardrails. Indeed, LLMs should never generate content promoting or normalizing harmful, illegal, or unethical behavior that may contribute to harm to individuals or society. This principle applies to both normal and adversarial use. In response, we introduce ALERT, a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It is designed to evaluate the …

abstract alert arxiv behavior benchmark building cs.cl cs.cy cs.lg generate guardrails harm indeed language language models large language large language models llms mind protect red teaming safety society them through type