Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models. (arXiv:2202.04557v2 [cs.NE] UPDATED)

Web: http://arxiv.org/abs/2202.04557

A large number of neural network models of associative memory have been
proposed in the literature. These include the classical Hopfield networks
(HNs), sparse distributed memories (SDMs), and more recently the modern
continuous Hopfield networks (MCHNs), which possesses close links with
self-attention in machine learning. In this paper, we propose a general
framework for understanding the operation of such memory networks as a sequence
of three operations: similarity, separation, and projection. We derive all
these memory models as instances of …

arxiv framework general memory models networks