Deep Stable neural networks: large-width asymptotics and convergence rates. (arXiv:2108.02316v2 [cs.LG] UPDATED)

In modern deep learning, there is a recent and growing literature on the
interplay between large-width asymptotic properties of deep Gaussian neural
networks (NNs), i.e. deep NNs with Gaussian-distributed weights, and Gaussian
stochastic processes (SPs). Such an interplay has proved to be critical in
Bayesian inference under Gaussian SP priors, kernel regression for infinitely
wide deep NNs trained via gradient descent, and information propagation within
infinitely wide NNs. Motivated by empirical analyses that show the potential of
replacing Gaussian distributions …

arxiv convergence lg networks neural networks