Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge

arXiv:2304.00891v2 Announce Type: replace
Abstract: We consider a resource-constrained Edge Device (ED), such as an IoT sensor or a microcontroller unit, embedded with a small-size ML model (S-ML) for a generic classification application and an Edge Server (ES) that hosts a large-size ML model (L-ML). Since the inference accuracy of S-ML is lower than that of the L-ML, offloading all the data samples to the ES results in high inference accuracy, but it defeats the purpose of embedding S-ML on …

abstract algorithms application applications arxiv classification cs.cv cs.lg deep learning edge embedded hierarchical inference iot microcontroller sensor server small the edge type