MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization

arXiv:2311.06798v2 Announce Type: replace
Abstract: Mixed-precision quantization of efficient networks often suffer from activation instability encountered in the exploration of bit selections. To address this problem, we propose a novel method called MetaMix which consists of bit selection and weight training phases. The bit selection phase iterates two steps, (1) the mixed-precision-aware weight update, and (2) the bit-search training with the fixed mixed-precision-aware weights, both of which combined reduce activation instability in mixed-precision quantization and contribute to fast and high-quality …

abstract arxiv cs.cv cs.lg exploration meta mixed mixed-precision networks novel precision quantization state training type