Dynamical stability and chaos in artificial neural network trajectories along training

arXiv:2404.05782v1 Announce Type: new
Abstract: The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally interpreted as a trajectory in network space -- a time series of networks -- and thus the training algorithm (e.g. gradient descent optimization of a suitable loss function) can be interpreted as a dynamical system in graph space. In order …

abstract artificial arxiv change chaos cond-mat.dis-nn cs.lg error interpreted iterative network neural network nlin.cd parameters physics.data-an prediction process space stability training trajectory type