The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks

arXiv:2405.10928v1 Announce Type: new
Abstract: Mechanistic interpretability aims to understand the behavior of neural networks by reverse-engineering their internal computations. However, current methods struggle to find clear interpretations of neural network activations because a decomposition of activations into computational features is missing. Individual neurons or model components do not cleanly correspond to distinct features or functions. We present a novel interpretability method that aims to overcome this limitation by transforming the activations of the network into a new basis - …

abstract arxiv behavior clear components computational cs.lg current engineering features however individual neurons interpretability network networks neural network neural networks neurons reverse-engineering struggle type