The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks. (arXiv:2110.03922v4 [cs.LG] UPDATED)

We derive a simple unified framework giving closed-form estimates for the
test risk and other generalization metrics of kernel ridge regression (KRR).
Relative to prior work, our derivations are greatly simplified and our final
expressions are more readily interpreted. These improvements are enabled by our
identification of a sharp conservation law which limits the ability of KRR to
learn any orthonormal basis of functions. Test risk and other objects of
interest are expressed transparently in terms of our conserved quantity …

arxiv conservation framework kernel law networks neural networks perspective regression