Understanding Distributed Representations of Concepts in Deep Neural Networks without Supervision

arXiv:2312.17285v2 Announce Type: replace-cross
Abstract: Understanding intermediate representations of the concepts learned by deep learning classifiers is indispensable for interpreting general model behaviors. Existing approaches to reveal learned concepts often rely on human supervision, such as pre-defined concept sets or segmentation processes. In this paper, we propose a novel unsupervised method for discovering distributed representations of concepts by selecting a principal subset of neurons. Our empirical findings demonstrate that instances with similar neuron activation states tend to share coherent concepts. …

abstract arxiv classifiers concept concepts cs.ai cs.cv cs.lg deep learning distributed general human intermediate networks neural networks novel paper processes segmentation supervision type understanding