Scaling and renormalization in high-dimensional regression

arXiv:2405.00592v1 Announce Type: cross
Abstract: This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models using the basic tools of random matrix theory and free probability. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning. Analytic formulas for the training and generalization errors are obtained in a few lines of algebra directly from the properties of the …

abstract arxiv basic cond-mat.dis-nn cs.lg derivation free introduction matrix paper performance probability random readers regression results review ridge scaling stat.ml theory tools topics training type