LLM Inference Unveiled: Survey and Roofline Model Insights

arXiv:2402.16363v1 Announce Type: new
Abstract: The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on …

abstract arxiv blend challenges clear cs.ai cs.cl framework inference insights language language model large language large language model llm opportunities presenting survey type understanding