On the Origins of Linear Representations in Large Language Models

arXiv:2403.03867v1 Announce Type: cross
Abstract: Recent works have argued that high-level semantic concepts are encoded "linearly" in the representation space of large language models. In this work, we study the origins of such linear representations. To that end, we introduce a simple latent variable model to abstract and formalize the concept dynamics of the next token prediction. We use this formalism to show that the next token prediction objective (softmax with cross-entropy) and the implicit bias of gradient descent together …

abstract arxiv concepts cs.cl cs.lg language language models large language large language models latent variable model linear representation semantic simple space stat.ml study type work