An Exact Kernel Equivalence for Finite Classification Models. (arXiv:2308.00824v3 [cs.LG] UPDATED)

We explore the equivalence between neural networks and kernel methods by
deriving the first exact representation of any finite-size parametric
classification model trained with gradient descent as a kernel machine. We
compare our exact representation to the well-known Neural Tangent Kernel (NTK)
and discuss approximation error relative to the NTK and other non-exact path
kernel formulations. We experimentally demonstrate that the kernel can be
computed for realistic networks up to machine precision. We use this exact
kernel to show that …

approximation arxiv classification classification model discuss error explore gradient kernel machine networks neural networks parametric representation