Mitigating Exaggerated Safety in Large Language Models

arXiv:2405.05418v1 Announce Type: new
Abstract: As the popularity of Large Language Models (LLMs) grow, combining model safety with utility becomes increasingly important. The challenge is making sure that LLMs can recognize and decline dangerous prompts without sacrificing their ability to be helpful. The problem of "exaggerated safety" demonstrates how difficult this can be. To reduce excessive safety behaviours -- which was discovered to be 26.1% of safe prompts being misclassified as dangerous and refused -- we use a combination of …

abstract arxiv challenge cs.cl language language models large language large language models llms making prompts safety type utility