NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models

arXiv:2402.09773v1 Announce Type: new
Abstract: The considerable size of Large Language Models (LLMs) presents notable deployment challenges, particularly on resource-constrained hardware. Structured pruning, offers an effective means to compress LLMs, thereby reducing storage costs and enhancing inference speed for more efficient utilization. In this work, we study data-efficient and resource-efficient structure pruning methods to obtain smaller yet still powerful models. Knowledge Distillation is well-suited for pruning, as the intact model can serve as an excellent teacher for pruned students. However, …

abstract arxiv challenges costs cs.cl data deployment hardware inference language language models large language large language models llms pruning speed storage storage costs study teachers type work