MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning

arXiv:2110.15352v2 Announce Type: replace
Abstract: Tiny deep learning on microcontroller units (MCUs) is challenging due to the limited memory size. We find that the memory bottleneck is due to the imbalanced memory distribution in convolutional neural network (CNN) designs: the first several blocks have an order of magnitude larger memory usage than the rest of the network. To alleviate this issue, we propose a generic patch-by-patch inference scheduling, which operates only on a small spatial region of the feature map …

abstract arxiv cnn convolutional neural network cs.cv deep learning designs distribution inference mcus memory microcontroller network neural network type units usage