Effective Learning with Node Perturbation in Deep Neural Networks

arXiv:2310.00965v3 Announce Type: replace
Abstract: Backpropagation (BP) is the dominant and most successful method for training parameters of deep neural network models. However, BP relies on two computationally distinct phases, does not provide a satisfactory explanation of biological learning, and can be challenging to apply for training of networks with discontinuities or noisy node dynamics. By comparison, node perturbation (NP) proposes learning by the injection of noise into network activations, and subsequent measurement of the induced loss change. NP relies …

abstract apply arxiv backpropagation cs.lg deep neural network however network networks neural network neural networks node parameters training type