Approximated Likelihood Ratio: A Forward-Only and Parallel Framework for Boosting Neural Network Training

arXiv:2403.12320v1 Announce Type: new
Abstract: Efficient and biologically plausible alternatives to backpropagation in neural network training remain a challenge due to issues such as high computational complexity and additional assumptions about neural networks, which limit scalability to deeper networks. The likelihood ratio method offers a promising gradient estimation strategy but is constrained by significant memory consumption, especially when deploying multiple copies of data to reduce estimation variance. In this paper, we introduce an approximation technique for the likelihood ratio (LR) …

abstract arxiv assumptions backpropagation boosting challenge complexity computational cs.ai cs.lg framework gradient likelihood network networks network training neural network neural networks scalability training type