Out-of-Distribution Detection Should Use Conformal Prediction (and Vice-versa?)

arXiv:2403.11532v1 Announce Type: cross
Abstract: Research on Out-Of-Distribution (OOD) detection focuses mainly on building scores that efficiently distinguish OOD data from In Distribution (ID) data. On the other hand, Conformal Prediction (CP) uses non-conformity scores to construct prediction sets with probabilistic coverage guarantees. In this work, we propose to use CP to better assess the efficiency of OOD scores. Specifically, we emphasize that in standard OOD benchmark settings, evaluation metrics can be overly optimistic due to the finite sample size …

abstract arxiv building conformity construct coverage cs.cv cs.lg data detection distribution prediction research stat.ml type work