TExplain: Explaining Learned Visual Features via Pre-trained (Frozen) Language Models

arXiv:2309.00733v4 Announce Type: replace-cross
Abstract: Interpreting the learned features of vision models has posed a longstanding challenge in the field of machine learning. To address this issue, we propose a novel method that leverages the capabilities of language models to interpret the learned features of pre-trained image classifiers. Our method, called TExplain, tackles this task by training a neural network to establish a connection between the feature space of image classifiers and language models. Then, during inference, our approach generates …

abstract arxiv capabilities challenge classifiers cs.cv cs.lg features image issue language language models machine machine learning novel type via vision vision models visual