Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input

arXiv:2402.08948v1 Announce Type: new
Abstract: In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is standard Gaussian and the output only depends on the projection of the input onto a low-dimensional subspace. We propose a basis-free generalization of the merged-staircase property in Abbe et al. (2022) and establish a necessary condition for the SGD-learnability. In addition, we prove that the condition is almost sufficient, in …

abstract analysis arxiv cs.lg distribution flow free gradient layer low mean networks neural networks projection standard stochastic study type work