Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime. (arXiv:2205.06798v1 [cs.LG])

The generalization performance of kernel ridge regression (KRR) exhibits a
multi-phased pattern that crucially depends on the scaling relationship between
the sample size $n$ and the underlying dimension $d$. This phenomenon is due to
the fact that KRR sequentially learns functions of increasing complexity as the
sample size increases; when $d^{k-1}\ll n\ll d^{k}$, only polynomials with
degree less than $k$ are learned. In this paper, we present sharp asymptotic
characterization of the performance of KRR at the critical transition regions …

arxiv kernel linear regression ridge