Gradient-based Automatic Per-Weight Mixed Precision Quantization for Neural Networks On-Chip

arXiv:2405.00645v1 Announce Type: new
Abstract: Model size and inference speed at deployment time, are major challenges in many deep learning applications. A promising strategy to overcome these challenges is quantization. However, a straightforward uniform quantization to very low precision can result in significant accuracy loss. Mixed-precision quantization, based on the idea that certain parts of the network can accommodate lower precision without compromising performance compared to other parts, offers a potential solution. In this work, we present High Granularity Quantization …

abstract accuracy applications arxiv challenges chip cs.lg deep learning deployment gradient however inference loss low major mixed mixed-precision networks neural networks per physics.ins-det precision quantization speed strategy type uniform