Lite-Mind: Towards Efficient and Robust Brain Representation Network

arXiv:2312.03781v3 Announce Type: replace
Abstract: The limited data availability and the low signal-to-noise ratio of fMRI signals lead to the challenging task of fMRI-to-image retrieval. State-of-the-art MindEye remarkably improves fMRI-to-image retrieval performance by leveraging a large model, i.e., a 996M MLP Backbone per subject, to align fMRI embeddings to the final hidden layer of CLIP's Vision Transformer (ViT). However, significant individual variations exist among subjects, even under identical experimental setups, mandating the training of large subject-specific models. The substantial parameters …

abstract art arxiv availability brain cs.ai cs.cv data embeddings fmri hidden image low mind mlp network noise per performance representation retrieval robust signal state type