Compression-aware Training of Neural Networks using Frank-Wolfe. (arXiv:2205.11921v1 [cs.LG])

Many existing Neural Network pruning approaches either rely on retraining to
compensate for pruning-caused performance degradation or they induce strong
biases to converge to a specific sparse solution throughout training. A third
paradigm obtains a wide range of compression ratios from a single dense
training run while also avoiding retraining. Recent work of Pokutta et al.
(2020) and Miao et al. (2022) suggests that the Stochastic Frank-Wolfe (SFW)
algorithm is particularly suited for training state-of-the-art models that are
robust to …

arxiv compression networks neural networks training