A Unified Kernel for Neural Network Learning

arXiv:2403.17467v1 Announce Type: new
Abstract: Past decades have witnessed a great interest in the distinction and connection between neural network learning and kernel learning. Recent advancements have made theoretical progress in connecting infinite-wide neural networks and Gaussian processes. Two predominant approaches have emerged: the Neural Network Gaussian Process (NNGP) and the Neural Tangent Kernel (NTK). The former, rooted in Bayesian inference, represents a zero-order kernel, while the latter, grounded in the tangent space of gradient descents, is a first-order kernel. …

abstract arxiv cs.ai cs.lg gaussian processes kernel network networks neural network neural networks process processes progress type