Cost-Effective Online Contextual Model Selection. (arXiv:2207.06030v2 [cs.LG] UPDATED)

How can we collect the most useful labels to learn a model selection policy,
when presented with arbitrary heterogeneous data streams? In this paper, we
formulate this task as an online contextual active model selection problem,
where at each round the learner receives an unlabeled data point along with a
context. The goal is to output the best model for any given context without
obtaining an excessive amount of labels. In particular, we focus on the task of
selecting pre-trained …

arxiv cost model selection