A study of uncertainty quantification in overparametrized high-dimensional models. (arXiv:2210.12760v2 [cs.LG] UPDATED)

Uncertainty quantification is a central challenge in reliable and trustworthy
machine learning. Naive measures such as last-layer scores are well-known to
yield overconfident estimates in the context of overparametrized neural
networks. Several methods, ranging from temperature scaling to different
Bayesian treatments of neural networks, have been proposed to mitigate
overconfidence, most often supported by the numerical observation that they
yield better calibrated uncertainty measures. In this work, we provide a sharp
comparison between popular uncertainty measures for binary classification in …

arxiv bayesian binary challenge classification comparison context machine machine learning networks neural networks numerical observation popular quantification scaling study trustworthy uncertainty work