Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Rates. (arXiv:2205.09860v2 [cs.LG] UPDATED)

We consider optimizing two-layer neural networks in the mean-field regime
where the learning dynamics of network weights can be approximated by the
evolution in the space of probability measures over the weight parameters
associated with the neurons. The mean-field regime is a theoretically
attractive alternative to the NTK (lazy training) regime which is only
restricted locally in the so-called neural tangent kernel space around
specialized initializations. Several prior works (\cite{chizat2018global,
mei2018mean}) establish the asymptotic global optimality of the mean-field
regime, …

analysis arxiv convergence global linear mean networks neural networks