Better Uncertainty Calibration via Proper Scores for Classification and Beyond

arXiv:2203.07835v4 Announce Type: replace
Abstract: With model trustworthiness being crucial for sensitive real-world applications, practitioners are putting more and more focus on improving the uncertainty calibration of deep neural networks. Calibration errors are designed to quantify the reliability of probabilistic predictions but their estimators are usually biased and inconsistent. In this work, we introduce the framework of proper calibration errors, which relates every calibration error to a proper score and provides a respective upper bound with optimal estimation properties. This …

abstract applications arxiv beyond classification cs.lg errors focus networks neural networks predictions reliability stat.ml type uncertainty via world