Proximal Mean Field Learning in Shallow Neural Networks. (arXiv:2210.13879v1 [cs.LG])

Recent mean field interpretations of learning dynamics in over-parameterized
neural networks offer theoretical insights on the empirical success of first
order optimization algorithms in finding global minima of the nonconvex risk
landscape. In this paper, we explore applying mean field learning dynamics as a
computational algorithm, rather than as an analytical tool. Specifically, we
design a Sinkhorn regularized proximal algorithm to approximate the
distributional flow from the learning dynamics in the mean field regime over
weighted point clouds. In this …

arxiv mean networks neural networks