Training Bayesian Neural Networks with Sparse Subspace Variational Inference

arXiv:2402.11025v1 Announce Type: new
Abstract: Bayesian neural networks (BNNs) offer uncertainty quantification but come with the downside of substantially increased training and inference costs. Sparse BNNs have been investigated for efficient inference, typically by either slowly introducing sparsity throughout the training or by post-training compression of dense BNNs. The dilemma of how to cut down massive training costs remains, particularly given the requirement to learn about the uncertainty. To solve this challenge, we introduce Sparse Subspace Variational Inference (SSVI), the …

abstract arxiv bayesian compression costs cs.lg inference inference costs networks neural networks quantification sparsity stat.ml training type uncertainty