Saliency Suppressed, Semantics Surfaced: Visual Transformations in Neural Networks and the Brain

arXiv:2404.18772v1 Announce Type: new
Abstract: Deep learning algorithms lack human-interpretable accounts of how they transform raw visual input into a robust semantic understanding, which impedes comparisons between different architectures, training objectives, and the human brain. In this work, we take inspiration from neuroscience and employ representational approaches to shed light on how neural networks encode information at low (visual saliency) and high (semantic similarity) levels of abstraction. Moreover, we introduce a custom image dataset where we systematically manipulate salient and …

abstract algorithms architectures arxiv brain cs.ai cs.cv deep learning deep learning algorithms human inspiration networks neural networks neuroscience raw robust semantic semantics training type understanding visual work