Towards a tailored mixed-precision sub-8bit quantization scheme for Gated Recurrent Units using Genetic Algorithms

arXiv:2402.12263v1 Announce Type: new
Abstract: Despite the recent advances in model compression techniques for deep neural networks, deploying such models on ultra-low-power embedded devices still proves challenging. In particular, quantization schemes for Gated Recurrent Units (GRU) are difficult to tune due to their dependence on an internal state, preventing them from fully benefiting from sub-8bit quantization. In this work, we propose a modular integer quantization scheme for GRUs where the bit width of each operator can be selected independently. We …

8bit abstract advances algorithms arxiv compression cs.lg cs.ne devices eess.sp embedded embedded devices gru low mixed mixed-precision networks neural networks power precision quantization type units