Fair Ranking under Disparate Uncertainty

arXiv:2309.01610v2 Announce Type: replace
Abstract: Ranking is a ubiquitous method for focusing the attention of human evaluators on a manageable subset of options. Its use as part of human decision-making processes ranges from surfacing potentially relevant products on an e-commerce site to prioritizing college applications for human review. While ranking can make human evaluation more effective by focusing attention on the most promising options, we argue that it can introduce unfairness if the uncertainty of the underlying relevance model differs …

abstract applications arxiv attention college commerce cs.cy cs.ir cs.lg decision e-commerce evaluation fair human making part processes products ranking review type uncertainty