Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap. (arXiv:2201.11676v3 [cs.LG] UPDATED)

Monitoring machine learning models once they are deployed is challenging. It
is even more challenging to decide when to retrain models in real-case
scenarios when labeled data is beyond reach, and monitoring performance metrics
becomes unfeasible. In this work, we use non-parametric bootstrapped
uncertainty estimates and SHAP values to provide explainable uncertainty
estimation as a technique that aims to monitor the deterioration of machine
learning models in deployment environments, as well as determine the source of
model deterioration when target …

arxiv bootstrap monitoring non-parametric parametric uncertainty