Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data

arXiv:2302.00834v2 Announce Type: replace
Abstract: We study the interpolation power of deep ReLU neural networks. Specifically, we consider the question of how efficiently, in terms of the number of parameters, deep ReLU networks can interpolate values at $N$ datapoints in the unit ball which are separated by a distance $\delta$. We show that $\Omega(N)$ parameters are required in the regime where $\delta$ is exponentially small in $N$, which gives the sharp result in this regime since $O(N)$ parameters are always …

abstract arxiv cs.lg cs.ne data networks neural networks parameters power question relu stat.ml study terms type values