FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU

The high computational and memory requirements of large language model
(LLM) inference make it feasible only with multiple high-end accelerators.
Motivated by the emerging demand for latency-insensitive tasks with batched
processing, this paper initiates the study of high-throughput LLM inference
using limited resources, such as a single commodity GPU.

accelerators computational demand generative gpu inference language language model language models large language large language model large language models latency llm memory multiple paper processing requirements research resources study