Training of Neural Networks with Uncertain Data -- A Mixture of Experts Approach

arXiv:2312.08083v2 Announce Type: replace-cross
Abstract: This paper introduces the "Uncertainty-aware Mixture of Experts" (uMoE), a novel solution aimed at addressing aleatoric uncertainty within Neural Network (NN) based predictive models. While existing methodologies primarily concentrate on managing uncertainty during inference, uMoE uniquely embeds uncertainty into the training phase. Employing a "Divide and Conquer" strategy, uMoE strategically partitions the uncertain input space into more manageable subspaces. It comprises Expert components, individually trained on their respective subspace uncertainties. Overarching the Experts, a Gating …

abstract arxiv cs.lg data experts inference mixture of experts network networks neural network neural networks novel paper predictive predictive models solution stat.ml training type uncertain uncertainty