Measuring Feature Dependency of Neural Networks by Collapsing Feature Dimensions in the Data Manifold

arXiv:2404.12341v1 Announce Type: new
Abstract: This paper introduces a new technique to measure the feature dependency of neural network models. The motivation is to better understand a model by querying whether it is using information from human-understandable features, e.g., anatomical shape, volume, or image texture. Our method is based on the principle that if a model is dependent on a feature, then removal of that feature should significantly harm its performance. A targeted feature is "removed" by collapsing the dimension …

abstract arxiv cs.cv cs.lg data dimensions feature features human image information manifold measuring motivation network networks neural network neural networks paper texture type