Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression. (arXiv:2204.10425v1 [math.ST])

We study the spectrum of inner-product kernel matrices, i.e., $n \times n$
matrices with entries $h (\langle \textbf{x}_i ,\textbf{x}_j \rangle/d)$ where
the $( \textbf{x}_i)_{i \leq n}$ are i.i.d.~random covariates in
$\mathbb{R}^d$. In the linear high-dimensional regime $n \asymp d$, it was
shown that these matrices are well approximated by their linearization, which
simplifies into the sum of a rescaled Wishart matrix and identity matrix. In
this paper, we generalize this decomposition to the polynomial high-dimensional
regime $n \asymp d^\ell,\ell \in …

arxiv kernel math polynomial product regression ridge