Target alignment in truncated kernel ridge regression. (arXiv:2206.14255v1 [cs.LG])

Kernel ridge regression (KRR) has recently attracted renewed interest due to
its potential for explaining the transient effects, such as double descent,
that emerge during neural network training. In this work, we study how the
alignment between the target function and the kernel affects the performance of
the KRR. We focus on the truncated KRR (TKRR) which utilizes an additional
parameter that controls the spectral truncation of the kernel matrix. We show
that for polynomial alignment, there is an \emph{over-aligned} …

alignment arxiv kernel lg regression ridge