Fairness and Unfairness in Binary and Multiclass Classification: Quantifying, Calculating, and Bounding

arXiv:2206.03234v2 Announce Type: replace
Abstract: We propose a new interpretable measure of unfairness, that allows providing a quantitative analysis of classifier fairness, beyond a dichotomous fair/unfair distinction. We show how this measure can be calculated when the classifier's conditional confusion matrices are known. We further propose methods for auditing classifiers for their fairness when the confusion matrices cannot be obtained or even estimated. Our approach lower-bounds the unfairness of a classifier based only on aggregate statistics, which may be provided …

arxiv binary classification cs.cy cs.lg fairness stat.ml type