Temporal Decisions: Leveraging Temporal Correlation for Efficient Decisions in Early Exit Neural Networks

arXiv:2403.07958v1 Announce Type: new
Abstract: Deep Learning is becoming increasingly relevant in Embedded and Internet-of-things applications. However, deploying models on embedded devices poses a challenge due to their resource limitations. This can impact the model's inference accuracy and latency. One potential solution are Early Exit Neural Networks, which adjust model depth dynamically through additional classifiers attached between their hidden layers. However, the real-time termination decision mechanism is critical for the system's efficiency, latency, and sustained accuracy.
This paper introduces Difference …

abstract accuracy applications arxiv challenge correlation cs.ai cs.lg decisions deep learning devices embedded embedded devices exit however impact inference internet latency limitations networks neural networks solution temporal type