Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective

arXiv:2403.14917v1 Announce Type: new
Abstract: In this paper, we study the feature learning ability of two-layer neural networks in the mean-field regime through the lens of kernel methods. To focus on the dynamics of the kernel induced by the first layer, we utilize a two-timescale limit, where the second layer moves much faster than the first layer. In this limit, the learning problem is reduced to the minimization problem over the intrinsic kernel. Then, we show the global convergence of …

abstract analysis arxiv cs.lg dynamics feature focus kernel layer mean networks neural networks paper perspective study through timescale type