Do language models plan ahead for future tokens?

arXiv:2404.00859v1 Announce Type: new
Abstract: Do transformers "think ahead" during inference at a given position? It is known transformers prepare information in the hidden states of the forward pass at $t$ that is then used in future forward passes $t+\tau$. We posit two explanations for this phenomenon: pre-caching, in which off-diagonal gradient terms present in training result in the model computing features at $t$ irrelevant to the present inference task but useful for the future, and breadcrumbs, in which features …

abstract arxiv caching cs.cl cs.lg future gradient hidden inference information language language models posit think tokens transformers type