DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation. (arXiv:2209.10797v1 [eess.SY] CROSS LISTED)

Transformer is a deep learning language model widely used for natural
language processing (NLP) services in datacenters. Among transformer models,
Generative Pre-trained Transformer (GPT) has achieved remarkable performance in
text generation, or natural language generation (NLG), which needs the
processing of a large input context in the summarization stage, followed by the
generation stage that produces a single word at a time. The conventional
platforms such as GPU are specialized for the parallel processing of large
inputs in the summarization …

arxiv latency text text generation transformer