Estimating Model Performance Under Covariate Shift Without Labels

arXiv:2401.08348v2 Announce Type: replace
Abstract: Machine learning models often experience performance degradation post-deployment due to shifts in data distribution. It is challenging to assess post-deployment performance accurately when labels are missing or delayed. Existing proxy methods, such as drift detection, fail to measure the effects of these shifts adequately. To address this, we introduce a new method for evaluating classification models on unlabeled data that accurately quantifies the impact of covariate shift on model performance and call it Probabilistic Adaptive …

abstract arxiv cs.lg data deployment detection distribution drift effects experience labels machine machine learning machine learning models performance replace shift type