Talking Heads: Understanding Inter-layer Communication in Transformer Language Models

arXiv:2406.09519v1 Announce Type: new
Abstract: Although it is known that transformer language models (LMs) pass features from early layers to later layers, it is not well understood how this information is represented and routed by the model. By analyzing particular mechanism LMs use to accomplish this, we find that it is also used to recall items from a list, and show that this mechanism can explain an otherwise arbitrary-seeming sensitivity of the model to the order of items in the …

abstract arxiv communication cs.ai cs.cl features information language language models later layer lms transformer transformer language models type understanding