From Empirical Observations to Universality: Dynamics of Deep Learning with Inputs Built on Gaussian mixture

arXiv:2405.00642v1 Announce Type: cross
Abstract: This study broadens the scope of theoretical frameworks in deep learning by delving into the dynamics of neural networks with inputs that demonstrate the structural characteristics to Gaussian Mixture (GM). We analyzed how the dynamics of neural networks under GM-structured inputs diverge from the predictions of conventional theories based on simple Gaussian structures. A revelation of our work is the observed convergence of neural network dynamics towards conventional theory even with standardized GM inputs, highlighting …

abstract arxiv cond-mat.dis-nn cond-mat.stat-mech cs.lg deep learning dynamics frameworks inputs networks neural networks stat.ml study type