Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs

arXiv:2402.10517v1 Announce Type: new
Abstract: Recently, considerable efforts have been directed towards compressing Large Language Models (LLMs), which showcase groundbreaking capabilities across diverse applications but entail significant deployment costs due to their large sizes. Meanwhile, much less attention has been given to mitigating the costs associated with deploying multiple LLMs of varying sizes despite its practical significance. Thus, this paper introduces \emph{any-precision LLM}, extending the concept of any-precision DNN to LLMs. Addressing challenges in any-precision LLM, we propose a lightweight …

abstract applications arxiv attention capabilities cost costs cs.lg deployment diverse diverse applications groundbreaking language language models large language large language models llm llms low multiple precision type