Helpfulness and Fairness of Task-Oriented Dialogue Systems. (arXiv:2205.12554v1 [cs.CL])

Task-oriented dialogue systems aim to answer questions from users and provide
immediate help. Therefore, how humans perceive their helpfulness is important.
However, neither the human-perceived helpfulness of task-oriented dialogue
systems nor its fairness implication has been studied yet. In this paper, we
define a dialogue response as helpful if it is relevant & coherent, useful, and
informative to a query and study computational measurements of helpfulness.
Then, we propose utilizing the helpfulness level of different groups to gauge
the fairness …

arxiv fairness systems