Accuracy-Preserving Calibration via Statistical Modeling on Probability Simplex

arXiv:2402.13765v1 Announce Type: new
Abstract: Classification models based on deep neural networks (DNNs) must be calibrated to measure the reliability of predictions. Some recent calibration methods have employed a probabilistic model on the probability simplex. However, these calibration methods cannot preserve the accuracy of pre-trained models, even those with a high classification accuracy. We propose an accuracy-preserving calibration method using the Concrete distribution as the probabilistic model on the probability simplex. We theoretically prove that a DNN model trained on …

abstract accuracy arxiv classification cs.lg modeling networks neural networks predictions pre-trained models probabilistic model probability reliability statistical statistical modeling stat.ml type via