Cost-Sensitive Learning to Defer to Multiple Experts with Workload Constraints

arXiv:2403.06906v1 Announce Type: new
Abstract: Learning to defer (L2D) aims to improve human-AI collaboration systems by learning how to defer decisions to humans when they are more likely to be correct than an ML classifier. Existing research in L2D overlooks key aspects of real-world systems that impede its practical adoption, namely: i) neglecting cost-sensitive scenarios, where type 1 and type 2 errors have different costs; ii) requiring concurrent human predictions for every instance of the training dataset and iii) not …

abstract adoption ai collaboration arxiv classifier collaboration constraints cost cs.ai cs.lg decisions experts human humans key multiple practical research systems type world