PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits

arXiv:2404.06453v1 Announce Type: cross
Abstract: The field of mechanistic interpretability aims to study the role of individual neurons in Deep Neural Networks. Single neurons, however, have the capability to act polysemantically and encode for multiple (unrelated) features, which renders their interpretation difficult. We present a method for disentangling polysemanticity of any Deep Neural Network by decomposing a polysemantic neuron into multiple monosemantic "virtual" neurons. This is achieved by identifying the relevant sub-graph ("circuit") for each "pure" feature. We demonstrate how …

arxiv circuits cs.ai cs.cv cs.lg features neurons type