Efficient Post-Training Augmentation for Adaptive Inference in Heterogeneous and Distributed IoT Environments

arXiv:2403.07957v1 Announce Type: new
Abstract: Early Exit Neural Networks (EENNs) present a solution to enhance the efficiency of neural network deployments. However, creating EENNs is challenging and requires specialized domain knowledge, due to the large amount of additional design choices. To address this issue, we propose an automated augmentation flow that focuses on converting an existing model into an EENN. It performs all required design decisions for the deployment to heterogeneous or distributed hardware targets: Our framework constructs the EENN …

abstract arxiv augmentation automated cs.ai cs.lg deployments design distributed domain domain knowledge efficiency environments exit however inference iot issue knowledge network networks neural network neural networks solution training type