On Calibration and Conformal Prediction of Deep Classifiers

In many classification applications, the prediction of a deep neural network (DNN) based classifier needs to be accompanied with some confidence indication. Two popular post-processing approaches for that aim are: 1) calibration: modifying the classifier's softmax values such that their maximum (associated with the prediction) better estimates the correctness probability; and 2) conformal prediction (CP): devising a score (based on the softmax values) from which a set of predictions with theoretically guaranteed marginal coverage of the correct class is produced. …

aim applications classification classifier classifiers confidence cs.lg deep neural network dnn network neural network popular post-processing prediction probability processing softmax stat.ml values