Approximation and interpolation of deep neural networks

arXiv:2304.10552v2 Announce Type: replace
Abstract: In this paper, we prove that in the overparametrized regime, deep neural network provide universal approximations and can interpolate any data set, as long as the activation function is locally in $L^1(\RR)$ and not an affine function.
Additionally, if the activation function is smooth and such an interpolation networks exists, then the set of parameters which interpolate forms a manifold. Furthermore, we give a characterization of the Hessian of the loss function evaluated at the …

abstract approximation arxiv cs.lg data data set deep neural network function interpolation math.oc math.pr network networks neural network neural networks paper prove set stat.ml type universal