E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity

arXiv:2310.15929v2 Announce Type: replace
Abstract: Traditional pruning methods are known to be challenging to work in Large Language Models (LLMs) for Generative AI because of their unaffordable training process and large computational demands. For the first time, we introduce the information entropy of hidden state features into a pruning metric design, namely E-Sparse, to improve the accuracy of N:M sparsity on LLM. E-Sparse employs the information richness to leverage the channel importance, and further incorporates several novel techniques to put …

abstract arxiv boosting computational cs.ai cs.cl cs.lg entropy features generative hidden inference information language language model language models large language large language model large language models llms process pruning sparsity state the information through training type work