Active learning with biased non-response to label requests

arXiv:2312.08150v2 Announce Type: replace
Abstract: Active learning can improve the efficiency of training prediction models by identifying the most informative new labels to acquire. However, non-response to label requests can impact active learning's effectiveness in real-world contexts. We conceptualise this degradation by considering the type of non-response present in the data, demonstrating that biased non-response is particularly detrimental to model performance. We argue that biased non-response is likely in contexts where the labelling process, by nature, relies on user interactions. …

abstract active learning arxiv cs.lg data efficiency however impact labels prediction prediction models stat.me stat.ml training type world