Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks

arXiv:2312.16020v3 Announce Type: replace
Abstract: This research embarks on pioneering the integration of gradient sampling optimization techniques, particularly StochGradAdam, into the pruning process of neural networks. Our main objective is to address the significant challenge of maintaining accuracy in pruned neural models, critical in resource-constrained scenarios. Through extensive experimentation, we demonstrate that gradient sampling significantly preserves accuracy during and after the pruning process compared to traditional optimization methods. Our study highlights the pivotal role of gradient sampling in robust learning …

abstract accuracy arxiv challenge cs.lg experimentation gradient integration networks neural networks optimization process pruning research residual robust sampling through type