Variational Bayesian Last Layers

arXiv:2404.11599v1 Announce Type: cross
Abstract: We introduce a deterministic variational formulation for training Bayesian last layer neural networks. This yields a sampling-free, single-pass model and loss that effectively improves uncertainty estimation. Our variational Bayesian last layer (VBLL) can be trained and evaluated with only quadratic complexity in last layer width, and is thus (nearly) computationally free to add to standard architectures. We experimentally investigate VBLLs, and show that they improve predictive accuracy, calibration, and out of distribution detection over baselines …

abstract arxiv bayesian complexity cs.cv cs.lg free layer loss networks neural networks sampling stat.ml training type uncertainty