ASPEST: Bridging the Gap Between Active Learning and Selective Prediction

arXiv:2304.03870v3 Announce Type: replace
Abstract: Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. These predictions can then be deferred to humans for further evaluation. As an everlasting challenge for machine learning, in many real-world scenarios, the distribution of test data is different from the training data. This results in more inaccurate predictions, and often increased dependence on humans, which can be difficult and expensive. Active learning aims to lower the overall labeling effort, …

abstract active learning arxiv challenge cs.lg data distribution evaluation gap humans learn machine machine learning making prediction predictions selective prediction test type uncertain world