Estimating Test Performance for AI Medical Devices under Distribution Shift with Conformal Prediction. (arXiv:2207.05796v1 [cs.LG])

Estimating the test performance of software AI-based medical devices under
distribution shifts is crucial for evaluating the safety, efficiency, and
usability prior to clinical deployment. Due to the nature of regulated medical
device software and the difficulty in acquiring large amounts of labeled
medical datasets, we consider the task of predicting the test accuracy of an
arbitrary black-box model on an unlabeled target domain without modification to
the original training process or any distributional assumptions of the original
source data …

ai ai medical arxiv devices distribution lg medical medical devices performance prediction shift test