Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions

arXiv:2402.15055v1 Announce Type: cross
Abstract: In this paper, we investigate the interplay between attention heads and specialized "next-token" neurons in the Multilayer Perceptron that predict specific tokens. By prompting an LLM like GPT-4 to explain these model internals, we can elucidate attention mechanisms that activate certain next-token neurons. Our analysis identifies attention heads that recognize contexts relevant to predicting a particular token, activating the associated neuron through the residual connection. We focus specifically on heads in earlier layers consistently activating …

abstract analysis arxiv attention attention mechanisms context cs.ai cs.cl cs.lg gpt gpt-4 interactions llm look mlp neurons next paper perceptron prompting token tokens transformers type ups