PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networks

arXiv:2404.00103v1 Announce Type: new
Abstract: Low-precision quantization is recognized for its efficacy in neural network optimization. Our analysis reveals that non-quantized elementwise operations which are prevalent in layers such as parameterized activation functions, batch normalization, and quantization scaling dominate the inference cost of low-precision models. These non-quantized elementwise operations are commonly overlooked in SOTA efficiency metrics such as Arithmetic Computation Effort (ACE). In this paper, we propose ACEv2 - an extended version of ACE which offers a better alignment with …

abstract analysis arxiv cost cs.cv cs.lg functions inference low network networks neural network neural networks normalization operations optimization precision quantization scaling type