From Zero to Hero: How local curvature at artless initial conditions leads away from bad minima

arXiv:2403.02418v1 Announce Type: new
Abstract: We investigate the optimization dynamics of gradient descent in a non-convex and high-dimensional setting, with a focus on the phase retrieval problem as a case study for complex loss landscapes. We first study the high-dimensional limit where both the number $M$ and the dimension $N$ of the data are going to infinity at fixed signal-to-noise ratio $\alpha = M/N$. By analyzing how the local curvature changes during optimization, we uncover that for intermediate $\alpha$, the …

abstract arxiv case case study cond-mat.dis-nn cond-mat.stat-mech cs.lg dynamics focus gradient leads loss optimization retrieval study type