Meta Pattern Concern Score: A Novel Evaluation Measure with Human Values for Multi-classifiers

arXiv:2209.06408v3 Announce Type: replace
Abstract: While advanced classifiers have been increasingly used in real-world safety-critical applications, how to properly evaluate the black-box models given specific human values remains a concern in the community. Such human values include punishing error cases of different severity in varying degrees and making compromises in general performance to reduce specific dangerous cases. In this paper, we propose a novel evaluation measure named Meta Pattern Concern Score based on the abstract representation of probabilistic prediction and …

abstract advanced applications arxiv box cases classifiers community cs.lg error evaluation human meta novel safety safety-critical type values world