Unsupervised Crowdsourcing with Accuracy and Cost Guarantees. (arXiv:2207.01988v1 [cs.LG])

We consider the problem of cost-optimal utilization of a crowdsourcing
platform for binary, unsupervised classification of a collection of items,
given a prescribed error threshold. Workers on the crowdsourcing platform are
assumed to be divided into multiple classes, based on their skill, experience,
and/or past performance. We model each worker class via an unknown confusion
matrix, and a (known) price to be paid per label prediction. For this setting,
we propose algorithms for acquiring label predictions from workers, and for …

accuracy arxiv cost crowdsourcing lg unsupervised