Finding Safety Neurons in Large Language Models

arXiv:2406.14144v1 Announce Type: new
Abstract: Large language models (LLMs) excel in various capabilities but also pose safety risks such as generating harmful content and misinformation, even after safety alignment. In this paper, we explore the inner mechanisms of safety alignment from the perspective of mechanistic interpretability, focusing on identifying and analyzing safety neurons within LLMs that are responsible for safety behaviors. We propose generation-time activation contrasting to locate these neurons and dynamic activation patching to evaluate their causal effects. Experiments …

abstract alignment arxiv capabilities cs.ai cs.cl cs.lg excel explore interpretability language language models large language large language models llms misinformation neurons paper perspective risks safety safety risks type