On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learning

arXiv:2311.12688v2 Announce Type: replace
Abstract: Bayesian deep learning and conformal prediction are two methods that have been used to convey uncertainty and increase safety in machine learning systems. We focus on combining Bayesian deep learning with split conformal prediction and how this combination effects out-of-distribution coverage; particularly in the case of multiclass image classification. We suggest that if the model is generally underconfident on the calibration set, then the resultant conformal sets may exhibit worse out-of-distribution coverage compared to simple …

abstract arxiv bayesian bayesian deep learning combination coverage cs.lg deep learning distribution effects focus learning systems machine machine learning prediction safety stat.ml systems type uncertainty