Structural Pruning of Pre-trained Language Models via Neural Architecture Search

arXiv:2405.02267v1 Announce Type: new
Abstract: Pre-trained language models (PLM), for example BERT or RoBERTa, mark the state-of-the-art for natural language understanding task when fine-tuned on labeled data. However, their large size poses challenges in deploying them for inference in real-world applications, due to significant GPU memory requirements and high inference latency. This paper explores neural architecture search (NAS) for structural pruning to find sub-parts of the fine-tuned network that optimally trade-off efficiency, for example in terms of model size or …

abstract applications architecture art arxiv bert challenges cs.cl cs.lg data example gpu however inference language language models language understanding mark memory natural natural language neural architecture search pruning requirements roberta search state them type understanding via world