Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition. (arXiv:2210.12256v1 [cs.LG])

Reliably estimating the uncertainty of a prediction throughout the model
lifecycle is crucial in many safety-critical applications.

The most common way to measure this uncertainty is via the predicted
confidence. While this tends to work well for in-domain samples, these
estimates are unreliable under domain drift.

Alternatively, a bias-variance decomposition allows to directly measure the
predictive uncertainty across the entire input space.

But, such a decomposition for proper scores does not exist in current
literature, and for exponential families it …

arxiv bias bias-variance general predictions uncertainty variance