The Positivity of the Neural Tangent Kernel

arXiv:2404.12928v1 Announce Type: new
Abstract: The Neural Tangent Kernel (NTK) has emerged as a fundamental concept in the study of wide Neural Networks. In particular, it is known that the positivity of the NTK is directly related to the memorization capacity of sufficiently wide networks, i.e., to the possibility of reaching zero loss in training, via gradient descent. Here we will improve on previous works and obtain a sharp result concerning the positivity of the NTK of feedforward networks of …

abstract arxiv capacity concept cs.ai cs.lg fundamental kernel loss math.pr math.sp networks neural networks possibility study type