DRIVE: Dual Gradient-Based Rapid Iterative Pruning

arXiv:2404.03687v1 Announce Type: new
Abstract: Modern deep neural networks (DNNs) consist of millions of parameters, necessitating high-performance computing during training and inference. Pruning is one solution that significantly reduces the space and time complexities of DNNs. Traditional pruning methods that are applied post-training focus on streamlining inference, but there are recent efforts to leverage sparsity early on by pruning before training. Pruning methods, such as iterative magnitude-based pruning (IMP) achieve up to a 90% parameter reduction while retaining accuracy comparable …

abstract arxiv complexities computing cs.cv cs.lg drive focus gradient inference iterative modern networks neural networks parameters performance pruning solution space space and time training type