Emergence of the SVD as an interpretable factorization in deep learning for inverse problems. (arXiv:2301.07820v2 [cs.LG] UPDATED)

Within the framework of deep learning we demonstrate the emergence of the
singular value decomposition (SVD) of the weight matrix as a tool for
interpretation of neural networks (NN) when combined with the descrambling
transformation--a recently-developed technique for addressing interpretability
in noisy parameter estimation neural networks \cite{amey2021neural}. By
considering the averaging effect of the data passed to the descrambling
minimization problem, we show that descrambling transformations--in the large
data limit--can be expressed in terms of the SVD of the NN …

arxiv deep learning emergence factorization framework interpretability interpretation matrix networks neural networks svd tool transformation value weight matrix