Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks

arXiv:2404.18769v1 Announce Type: cross
Abstract: Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks as the curse of dimensionality (CoD) cannot be evaded when trying to approximate even a single ReLU neuron (Bach, 2017). In this paper, we study a suitable function space for over-parameterized two-layer neural networks with bounded norms (e.g., the path norm, the Barron norm) in the perspective of sample complexity and generalization properties. First, …

abstract arxiv cs.lg dimensionality functions kernel layer networks neural networks neuron norm paper relu show space stat.ml studies study the curse of dimensionality type