A Survey on Efficient Inference for Large Language Models

arXiv:2404.14294v1 Announce Type: new
Abstract: Large Language Models (LLMs) have attracted extensive attention due to their remarkable performance across various tasks. However, the substantial computational and memory requirements of LLM inference pose challenges for deployment in resource-constrained scenarios. Efforts within the field have been directed towards developing techniques aimed at enhancing the efficiency of LLM inference. This paper presents a comprehensive survey of the existing literature on efficient LLM inference. We start by analyzing the primary causes of the inefficient …

abstract arxiv attention challenges computational cs.ai cs.cl deployment however inference language language models large language large language models llm llms memory performance requirements survey tasks type