OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization

arXiv:2403.12983v1 Announce Type: cross
Abstract: Structured pruning is a promising approach for reducing the inference costs of large vision and language models. By removing carefully chosen structures, e.g., neurons or attention heads, the improvements from this approach can be realized on standard deep learning hardware. In this work, we focus on structured pruning in the one-shot (post-training) setting, which does not require model retraining after pruning. We propose a novel combinatorial optimization framework for this problem, based on a layer-wise …

abstract arxiv attention costs cs.cv cs.lg deep learning hardware improvements inference inference costs language language models neurons optimization pruning standard type vision work