What does the Knowledge Neuron Thesis Have to do with Knowledge?

arXiv:2405.02421v1 Announce Type: new
Abstract: We reassess the Knowledge Neuron (KN) Thesis: an interpretation of the mechanism underlying the ability of large language models to recall facts from a training corpus. This nascent thesis proposes that facts are recalled from the training corpus through the MLP weights in a manner resembling key-value memory, implying in effect that "knowledge" is stored in the network. Furthermore, by modifying the MLP modules, one can control the language model's generation of factual information. The …

abstract arxiv cs.cl facts interpretation knowledge language language models large language large language models mlp neuron recall thesis through training type