Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference

arXiv:2312.15159v2 Announce Type: replace
Abstract: Recent advancements in large language models (LLMs) boasting billions of parameters have generated a significant demand for efficient deployment in inference workloads. The majority of existing approaches rely on temporal architectures that reuse hardware units for different network layers and operators. However, these methods often encounter challenges in achieving low latency due to considerable memory access overhead. This paper investigates the feasibility and potential of model-specific spatial acceleration for LLM inference on FPGAs. Our approach …

abstract architectures arxiv cs.ai cs.ar cs.cl cs.lg demand deployment fpga generated hardware however inference language language model language models large language large language model large language models llms network operators parameters spatial temporal type understanding units workloads