Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers

arXiv:2403.19591v1 Announce Type: new
Abstract: Non-linear functions are prevalent in Transformers and their lightweight variants, incurring substantial and frequently underestimated hardware costs. Previous state-of-the-art works optimize these operations by piece-wise linear approximation and store the parameters in look-up tables (LUT), but most of them require unfriendly high-precision arithmetics such as FP/INT 32 and lack consideration of integer-only INT quantization. This paper proposed a genetic LUT-Approximation algorithm namely GQA-LUT that can automatically determine the parameters with quantization awareness. The results demonstrate …

approximation arxiv cs.ar cs.lg cs.ne linear non-linear operations quantization transformers type