QuantProb: Generalizing Probabilities along with Predictions for a Pre-trained Classifier

arXiv:2304.12766v2 Announce Type: replace
Abstract: Quantification of Uncertainty in predictions is a challenging problem. In the classification settings, although deep learning based models generalize well, class probabilities often lack reliability. Calibration errors are used to quantify uncertainty, and several methods exist to minimize calibration error. We argue that between the choice of having a minimum calibration error on original distribution which increases across distortions or having a (possibly slightly higher) calibration error which is constant across distortions, we prefer the …

abstract arxiv calibration class classification classifier cs.lg deep learning error errors predictions quantification reliability type uncertainty