Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods

arXiv:2212.06872v4 Announce Type: replace
Abstract: In order to gain insights about the decision-making of different visual recognition backbones, we propose two methodologies, sub-explanation counting and cross-testing, that systematically applies deep explanation algorithms on a dataset-wide basis, and compares the statistics generated from the amount and nature of the explanations. These methodologies reveal the difference among networks in terms of two properties called compositionality and disjunctivism. Transformers and ConvNeXt are found to be more compositional, in the sense that they jointly …

abstract algorithms arxiv cnns cs.cv dataset decision generated insights making nature recognition statistics testing transformers type via visual