Frame Quantization of Neural Networks

arXiv:2404.08131v1 Announce Type: new
Abstract: We present a post-training quantization algorithm with error estimates relying on ideas originating from frame theory. Specifically, we use first-order Sigma-Delta ($\Sigma\Delta$) quantization for finite unit-norm tight frames to quantize weight matrices and biases in a neural network. In our scenario, we derive an error bound between the original neural network and the quantized neural network in terms of step size and the number of frame elements. We also demonstrate how to leverage the redundancy …

abstract algorithm arxiv biases cs.it cs.lg delta error ideas math.it network networks neural network neural networks norm quantization stat.ml theory training type