Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors. (arXiv:2201.11651v1 [cs.LG])

Web: http://arxiv.org/abs/2201.11651

Applications of neural networks on edge systems have proliferated in recent
years but the ever-increasing model size makes neural networks not able to
deploy on resource-constrained microcontrollers efficiently. We propose
bit-serial weight pools, an end-to-end framework that includes network
compression and acceleration of arbitrary sub-byte precision. The framework can
achieve up to 8x compression compared to 8-bit networks by sharing a pool of
weights across the entire network. We further propose a bit-serial lookup based
software implementation that allows runtime-bitwidth …

arxiv compression networks neural neural networks precision processors