DREAM: Visual Decoding from Reversing Human Visual System

arXiv:2310.02265v2 Announce Type: replace-cross
Abstract: In this work we present DREAM, an fMRI-to-image method for reconstructing viewed images from brain activities, grounded on fundamental knowledge of the human visual system. We craft reverse pathways that emulate the hierarchical and parallel nature of how humans perceive the visual world. These tailored pathways are specialized to decipher semantics, color, and depth cues from fMRI data, mirroring the forward pathways from visual stimuli to fMRI recordings. To do so, two components mimic the …

abstract arxiv brain craft cs.cv cs.lg decoding eess.iv fmri hierarchical human humans image images knowledge nature q-bio.nc type visual work world