Adaptive Sharpness-Aware Pruning for Robust Sparse Networks

arXiv:2306.14306v2 Announce Type: replace
Abstract: Robustness and compactness are two essential attributes of deep learning models that are deployed in the real world. The goals of robustness and compactness may seem to be at odds, since robustness requires generalization across domains, while the process of compression exploits specificity in one domain. We introduce Adaptive Sharpness-Aware Pruning (AdaSAP), which unifies these goals through the lens of network sharpness. The AdaSAP method produces sparse networks that are robust to input variations which …

abstract arxiv compression cs.cv cs.lg deep learning domain domains exploits networks process pruning robust robustness specificity type world