Multi-layer random features and the approximation power of neural networks

arXiv:2404.17461v1 Announce Type: new
Abstract: A neural architecture with randomly initialized weights, in the infinite width limit, is equivalent to a Gaussian Random Field whose covariance function is the so-called Neural Network Gaussian Process kernel (NNGP). We prove that a reproducing kernel Hilbert space (RKHS) defined by the NNGP contains only functions that can be approximated by the architecture. To achieve a certain approximation error the required number of neurons in each layer is defined by the RKHS norm of …

abstract approximation architecture arxiv covariance cs.ai cs.lg features function kernel layer network networks neural network neural networks power process prove random space type