What You See is What You Classify: Black Box Attributions. (arXiv:2205.11266v2 [cs.CV] UPDATED)

An important step towards explaining deep image classifiers lies in the
identification of image regions that contribute to individual class scores in
the model's output. However, doing this accurately is a difficult task due to
the black-box nature of such networks. Most existing approaches find such
attributions either using activations and gradients or by repeatedly perturbing
the input. We instead address this challenge by training a second deep network,
the Explainer, to predict attributions for a pre-trained black-box classifier,
the …

arxiv black box