A simple theory for training response of deep neural networks

arXiv:2405.04074v1 Announce Type: cross
Abstract: Deep neural networks give us a powerful method to model the training dataset's relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a whole. The network's behavior is training dynamics with a feedback loop from the evaluation of the loss function. We already know the training response can be constant or shows power law-like aging in some ideal …

abstract artificial arxiv behavior cond-mat.dis-nn cs.ai cs.lg dataset memory network networks neural networks neurons nlin.ao regard relationship simple theory training type work