Geometric structure of Deep Learning networks and construction of global ${\mathcal L}^2$ minimizers

arXiv:2309.10639v4 Announce Type: replace
Abstract: In this paper, we explicitly determine local and global minimizers of the $\mathcal{L}^2$ cost function in underparametrized Deep Learning (DL) networks; our main goal is to shed light on their geometric structure and properties. We accomplish this by a direct construction, without invoking the gradient descent flow at any point of this work. We specifically consider $L$ hidden layers, a ReLU ramp activation function, an $\mathcal{L}^2$ Schatten class (or Hilbert-Schmidt) cost function, input and output …

abstract arxiv construction cost cs.ai cs.lg deep learning function global light math.mp math.oc math-ph networks paper stat.ml type