Gradient-flow adaptive importance sampling for Bayesian leave one out cross-validation for sigmoidal classification models

We introduce a set of gradient-flow-guided adaptive importance sampling (IS) transformations to stabilize Monte-Carlo approximations of point-wise leave one out cross-validated (LOO) predictions for Bayesian classification models. One can leverage this methodology for assessing model generalizability by for instance computing a LOO analogue to the AIC or computing LOO ROC/PRC curves and derived metrics like the AUROC and AUPRC. By the calculus of variations and gradient flow, we derive two simple nonlinear single-step transformations that utilize gradient information to shift …

aic bayesian classification computing cs.ai cs.lg flow gradient importance instance math.sp math.st methodology monte-carlo predictions sampling set stat.me stat.th validation wise