DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit. (arXiv:2207.12551v1 [cs.CL])

Dialog system developers need high-quality data to train, fine-tune and
assess their systems. They often use crowdsourcing for this since it provides
large quantities of data from many workers. However, the data may not be of
sufficiently good quality. This can be due to the way that the requester
presents a task and how they interact with the workers. This paper introduces
DialCrowd 2.0 to help requesters obtain higher quality data by, for example,
presenting tasks more clearly and facilitating …

arxiv crowdsourcing quality toolkit