Asymptotics of Learning with Deep Structured (Random) Features

arXiv:2402.13999v1 Announce Type: cross
Abstract: For a large class of feature maps we provide a tight asymptotic characterisation of the test error associated with learning the readout layer, in the high-dimensional limit where the input dimension, hidden layer widths, and number of training samples are proportionally large. This characterization is formulated in terms of the population covariance of the features. Our work is partially motivated by the problem of learning with Gaussian rainbow neural networks, namely deep non-linear fully-connected networks …

abstract arxiv class cond-mat.dis-nn cs.lg error feature features hidden layer maps math.st random samples stat.ml stat.th test training type