BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity

arXiv:2310.04420v2 Announce Type: replace
Abstract: Understanding the functional organization of higher visual cortex is a central focus in neuroscience. Past studies have primarily mapped the visual and semantic selectivity of neural populations using hand-selected stimuli, which may potentially bias results towards pre-existing hypotheses of visual cortex functionality. Moving beyond conventional approaches, we introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. Our method -- Semantic Captioning Using Brain Alignments ("BrainSCUBA") …

abstract arxiv beyond bias captions cortex cs.lg fine-grained focus functional language mapped moving natural natural language neuroscience organization q-bio.nc results semantic studies type understanding visual visual cortex