Interpretability Needs a New Paradigm

arXiv:2405.05386v1 Announce Type: new
Abstract: Interpretability is the study of explaining models in understandable terms to humans. At present, interpretability is divided into two paradigms: the intrinsic paradigm, which believes that only models designed to be explained can be explained, and the post-hoc paradigm, which believes that black-box models can be explained. At the core of this debate is how each paradigm ensures its explanations are faithful, i.e., true to the model's behavior. This is important, as false but convincing …

abstract arxiv box core cs.cl cs.cv cs.lg explained humans interpretability intrinsic new paradigm paradigm stat.ml study terms type