Resource-Constrained Edge AI with Early Exit Prediction. (arXiv:2206.07269v1 [cs.LG])

Web: http://arxiv.org/abs/2206.07269

By leveraging the data sample diversity, the early-exit network recently
emerges as a prominent neural network architecture to accelerate the deep
learning inference process. However, intermediate classifiers of the early
exits introduce additional computation overhead, which is unfavorable for
resource-constrained edge artificial intelligence (AI). In this paper, we
propose an early exit prediction mechanism to reduce the on-device computation
overhead in a device-edge co-inference system supported by early-exit networks.
Specifically, we design a low-complexity module, namely the Exit Predictor, to …

ai arxiv edge edge ai exit lg prediction