Geometric structure of shallow neural networks and constructive ${\mathcal L}^2$ cost minimization

arXiv:2309.10370v2 Announce Type: replace
Abstract: In this paper, we approach the problem of cost (loss) minimization in underparametrized shallow neural networks through the explicit construction of upper bounds, without any use of gradient descent. A key focus is on elucidating the geometric structure of approximate and precise minimizers. We consider shallow neural networks with one hidden layer, a ReLU activation function, an ${\mathcal L}^2$ Schatten class (or Hilbert-Schmidt) cost function, input space ${\mathbb R}^M$, output space ${\mathbb R}^Q$ with $Q\leq …

abstract arxiv construction cost cs.ai cs.lg focus gradient key loss math.mp math.oc math-ph networks neural networks paper stat.ml through type