Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes

arXiv:2310.16597v3 Announce Type: replace-cross
Abstract: The infinitely wide neural network has been proven a useful and manageable mathematical model that enables the understanding of many phenomena appearing in deep learning. One example is the convergence of random deep networks to Gaussian processes that allows a rigorous analysis of the way the choice of activation function and network weights impacts the training dynamics. In this paper, we extend the seminal proof of Matthews et al. (2018) to a larger class of …

abstract analysis arxiv beyond convergence cs.lg deep learning example gaussian processes low network networks neural network neural networks processes random stat.ml type understanding