Fundamental Components of Deep Learning: A category-theoretic approach

arXiv:2403.13001v1 Announce Type: new
Abstract: Deep learning, despite its remarkable achievements, is still a young field. Like the early stages of many scientific disciplines, it is marked by the discovery of new phenomena, ad-hoc design decisions, and the lack of a uniform and compositional mathematical foundation. From the intricacies of the implementation of backpropagation, through a growing zoo of neural network architectures, to the new and poorly understood phenomena such as double descent, scaling laws or in-context learning, there are …

abstract arxiv components cs.ai cs.lg decisions deep learning design discovery foundation implementation math.ct scientific type uniform young