Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping

Much recent research effort has been devoted to explain the success of deep learning. Random Matrix Theory (RMT) provides an emerging way to this end by analyzing the spectra of large random matrices involved in a trained deep neural network (DNN) such as weight matrices or Hessian matrices in the stochastic gradient descent algorithm. To better understand spectra of weight matrices, we conduct extensive experiments on weight matrices under different settings for layers, networks and data sets. Based on the …

algorithm application classification data data sets deep learning deep neural network dnn early-stopping gradient impact matrix network networks neural network random research stochastic success theory work