Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

arXiv:2405.15071v1 Announce Type: new
Abstract: We study whether transformers can learn to implicitly reason over parametric knowledge, a skill that even the most capable language models struggle with. Focusing on two representative reasoning types, composition and comparison, we consistently find that transformers can learn implicit reasoning, but only through grokking, i.e., extended training far beyond overfitting. The levels of generalization also vary across reasoning types: when faced with out-of-distribution examples, transformers fail to systematically generalize for composition but succeed for …

abstract arxiv comparison cs.cl edge journey knowledge language language models learn parametric reason reasoning skill struggle study the edge transformers type types