Reusing Softmax Hardware Unit for GELU Computation in Transformers

arXiv:2402.10118v1 Announce Type: cross
Abstract: Transformers have improved drastically the performance of natural language processing (NLP) and computer vision applications. The computation of transformers involves matrix multiplications and non-linear activation functions such as softmax and GELU (Gaussion Error Linear Unit) that are accelerated directly in hardware. Currently, function evaluation is done separately for each function and rarely allows for hardware reuse. To mitigate this problem, in this work, we map the computation of GELU to a softmax operator. In this …

abstract applications arxiv computation computer computer vision cs.ar cs.lg error evaluation function functions hardware language language processing linear matrix natural natural language natural language processing nlp non-linear performance processing softmax transformers type vision