Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case. (arXiv:2201.04543v1 [stat.ML])

The paper deals with the distribution of singular values of the input-output
Jacobian of deep untrained neural networks in the limit of their infinite
width. The Jacobian is the product of random matrices where the independent
rectangular weight matrices alternate with diagonal matrices whose entries
depend on the corresponding column of the nearest neighbor weight matrix. The
problem was considered in \cite{Pe-Co:18} for the Gaussian weights and biases
and also for the weights that are Haar distributed orthogonal matrices and …

arxiv distribution ml networks neural networks random