Improving equilibrium propagation without weight symmetry through Jacobian homeostasis

arXiv:2309.02214v2 Announce Type: replace
Abstract: Equilibrium propagation (EP) is a compelling alternative to the backpropagation of error algorithm (BP) for computing gradients of neural networks on biological or analog neuromorphic substrates. Still, the algorithm requires weight symmetry and infinitesimal equilibrium perturbations, i.e., nudges, to estimate unbiased gradients efficiently. Both requirements are challenging to implement in physical systems. Yet, whether and how weight asymmetry affects its applicability is unknown because, in practice, it may be masked by biases introduced through the …

abstract algorithm analog arxiv backpropagation computing cs.ai cs.lg cs.ne equilibrium error improving networks neural networks neuromorphic propagation requirements symmetry the algorithm through type unbiased