Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN

arXiv:2310.09163v2 Announce Type: replace
Abstract: Large pretrained models, coupled with fine-tuning, are slowly becoming established as the dominant architecture in machine learning. Even though these models offer impressive performance, their practical application is often limited by the prohibitive amount of resources required for every inference. Early-exiting dynamic neural networks (EDNN) circumvent this issue by allowing a model to make some of its predictions from intermediate layers (i.e., early-exit). Training an EDNN architecture is challenging as it consists of two intertwined …

abstract application architecture arxiv cs.lg dnn dynamic every exit fine-tuning inference machine machine learning network networks neural network neural networks performance practical pretrained models replace resources type