QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted ensembles

arXiv:2404.12599v1 Announce Type: new
Abstract: Existing methods for uncertainty quantification incur massive memory and compute overhead, often requiring multiple models/inferences. Hence they are impractical on ultra-low-power KB-sized TinyML devices. To reduce overhead, prior works have proposed the use of early-exit networks as ensembles to quantify uncertainty in a single forward-pass. However, they still have a prohibitive cost for tinyML. To address these challenges, we propose QUTE, a novel resource-efficient early-exit-assisted ensemble architecture optimized for tinyML models. QUTE adds additional output …

abstract arxiv compute cs.cv cs.lg devices exit however inferences low massive memory multiple networks power prior quantification reduce tinyml type uncertainty