Spectral Analysis and Fixed Point Stability of Deep Neural Dynamics. (arXiv:2011.13492v2 [cs.LG] UPDATED)

In this work, we analyze the eigenvalue spectra and stability of
discrete-time dynamical systems parametrized by deep neural networks. In
particular, we leverage a representation of deep neural networks as pointwise
affine maps, thus exposing their local linear operators and making them
accessible to classical system analytic methods.The view of neural networks as
affine parameter varying maps allows us to "crack open the black box" of neural
network dynamical behavior by visualizing stationary points, state-space
partitioning, and eigenvalue spectra. We …

analysis arxiv