MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data

arXiv:2403.11207v1 Announce Type: new
Abstract: Reconstructions of visual perception from brain activity have improved tremendously, but the practical utility of such methods has been limited. This is because such models are trained independently per subject where each subject requires dozens of hours of expensive fMRI training data to attain high-quality results. The present work showcases high-quality reconstructions using only 1 hour of fMRI training data. We pretrain our model across 7 subjects and then fine-tune on minimal data from a …

abstract arxiv brain brain activity cs.ai cs.cv data fmri hour image per perception practical q-bio.nc quality training training data type utility visual