Model Compression and Efficient Inference for Large Language Models: A Survey

arXiv:2402.09748v1 Announce Type: cross
Abstract: Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it challenging to deploy large models on resource-constrained devices. In this paper, we investigate compression and efficient inference methods for large language models from an algorithmic perspective. Regarding taxonomy, similar to smaller models, compression and acceleration algorithms for large language models can still be categorized into quantization, pruning, distillation, compact architecture design, dynamic …

abstract arxiv compression computational costs cs.ai cs.cl cs.lg cs.pf deploy devices inference language language models large language large language models large models memory paper process success survey transformer type