An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression

arXiv:2306.13185v2 Announce Type: replace-cross
Abstract: We study the cost of overfitting in noisy kernel ridge regression (KRR), which we define as the ratio between the test error of the interpolating ridgeless model and the test error of the optimally-tuned model. We take an "agnostic" view in the following sense: we consider the cost as a function of sample size for any target function, even if the sample size is not large enough for consistency or the target is outside the …

abstract arxiv cost cs.lg error kernel overfitting regression ridge stat.ml study test type view