SySMOL: Co-designing Algorithms and Hardware for Neural Networks with Heterogeneous Precisions

arXiv:2311.14114v2 Announce Type: replace-cross
Abstract: Recent quantization techniques have enabled heterogeneous precisions at very fine granularity, e.g., each parameter/activation can take on a different precision, resulting in compact neural networks without sacrificing accuracy. However, there is a lack of efficient architectural support for such networks, which require additional hardware to decode the precision settings for individual variables, align the variables, and provide fine-grained mixed-precision compute capabilities. The complexity of these operations introduces high overheads. Thus, the improvements in inference latency/energy …

abstract accuracy algorithms arxiv compact cs.ar cs.lg cs.pf designing hardware however networks neural networks precision quantization quantization techniques support type