Fixing confirmation bias in feature attribution methods via semantic match

arXiv:2307.00897v2 Announce Type: replace
Abstract: Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about …

abstract arxiv attribution become behavior bias black box box concepts confirmation bias cs.ai cs.lg feature human interpretation match scholars semantic success terms type via