COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization

arXiv:2403.07134v1 Announce Type: new
Abstract: Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing these models to their low-bit counterparts without compromising the original accuracy remains a key challenge. In this paper, we propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. We consider the widely used integer quantization, where every quantized weight can be decomposed into a …

abstract accuracy algorithm arxiv backpropagation challenge cs.cv cs.lg deployment free however key low making networks neural networks paper practical quantization them training type