Explorations of Self-Repair in Language Models

arXiv:2402.15390v1 Announce Type: new
Abstract: Prior interpretability research studying narrow distributions has preliminarily identified self-repair, a phenomena where if components in large language models are ablated, later components will change their behavior to compensate. Our work builds off this past literature, demonstrating that self-repair exists on a variety of models families and sizes when ablating individual attention heads on the full training distribution. We further show that on the full training distribution self-repair is imperfect, as the original direct effect …

abstract arxiv behavior change components cs.ai cs.cl cs.lg families interpretability interpretability research language language models large language large language models literature narrow prior research studying type will work