Deep Learning with Parametric Lenses

arXiv:2404.00408v1 Announce Type: new
Abstract: We propose a categorical semantics for machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, and different architectures, shedding new light on their similarities and differences. Furthermore, our approach to learning has examples …

abstract adam algorithms arxiv categorical cs.lg cs.lo deep learning foundation framework gradient loss machine machine learning machine learning algorithms maps parametric semantics terms type