Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring. (arXiv:2206.08288v1 [cs.CL])

Short answer scoring (SAS) is the task of grading short text written by a
learner. In recent years, deep-learning-based approaches have substantially
improved the performance of SAS models, but how to guarantee high-quality
predictions still remains a critical issue when applying such models to the
education field. Towards guaranteeing high-quality predictions, we present the
first study of exploring the use of human-in-the-loop framework for minimizing
the grading cost while guaranteeing the grading quality by allowing a SAS model
to share …

arxiv cost exploration frameworks human loop quality scoring