Analyzing And Editing Inner Mechanisms Of Backdoored Language Models

arXiv:2302.12461v3 Announce Type: replace
Abstract: Poisoning of data sets is a potential security threat to large language models that can lead to backdoored models. A description of the internal mechanisms of backdoored language models and how they process trigger inputs, e.g., when switching to toxic language, has yet to be found. In this work, we study the internal representations of transformer-based backdoored language models and determine early-layer MLP modules as most important for the backdoor mechanism in combination with the …

abstract arxiv cs.ai cs.cl cs.lg data data sets editing found inputs language language models large language large language models process security threat toxic language type