The Efficiency Spectrum of Large Language Models: An Algorithmic Survey

arXiv:2312.00678v2 Announce Type: replace
Abstract: The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains, reshaping the artificial general intelligence landscape. However, the increasing computational and memory demands of these models present substantial challenges, hindering both academic research and practical applications. To address these issues, a wide array of methods, including both algorithmic and hardware solutions, have been developed to enhance the efficiency of LLMs. This survey delivers a comprehensive review of algorithmic …

arxiv cs.cl efficiency language language models large language large language models spectrum survey type