The phase diagram of kernel interpolation in large dimensions

arXiv:2404.12597v1 Announce Type: new
Abstract: The generalization ability of kernel interpolation in large dimensions (i.e., $n \asymp d^{\gamma}$ for some $\gamma>0$) might be one of the most interesting problems in the recent renaissance of kernel regression, since it may help us understand the 'benign overfitting phenomenon' reported in the neural networks literature. Focusing on the inner product kernel on the sphere, we fully characterized the exact order of both the variance and bias of large-dimensional kernel interpolation under various source …

abstract arxiv cs.lg dimensions interpolation kernel math.st networks neural networks overfitting regression stat.ml stat.th type