QGen: On the Ability to Generalize in Quantization Aware Training

arXiv:2404.11769v1 Announce Type: new
Abstract: Quantization lowers memory usage, computational requirements, and latency by utilizing fewer bits to represent model weights and activations. In this work, we investigate the generalization properties of quantized neural networks, a characteristic that has received little attention despite its implications on model performance. In particular, first, we develop a theoretical model for quantization in neural networks and demonstrate how quantization functions as a form of regularization. Second, motivated by recent work connecting the sharpness of …

abstract arxiv attention computational cs.cv cs.lg latency memory networks neural networks performance quantization requirements training type usage work