A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator

arXiv:2403.08938v1 Announce Type: cross
Abstract: We consider learning an unknown target function $f_*$ using kernel ridge regression (KRR) given i.i.d. data $(u_i,y_i)$, $i\leq n$, where $u_i \in U$ is a covariate vector and $y_i = f_* (u_i) +\varepsilon_i \in \mathbb{R}$. A recent string of work has empirically shown that the test error of KRR can be well approximated by a closed-form estimate derived from an `equivalent' sequence model that only depends on the spectrum of the kernel operator. However, a …

abstract arxiv cs.lg data error estimator function kernel math.st regression ridge stat.ml stat.th string test theory type vector work