Decoding Continuous Character-based Language from Non-invasive Brain Recordings

arXiv:2403.11183v1 Announce Type: new
Abstract: Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge. Previous non-invasive decoders either require multiple experiments with identical stimuli to pinpoint cortical regions and enhance signal-to-noise ratios in brain activity, or they are limited to discerning basic linguistic elements such as letters and words. We propose a novel approach to decoding continuous language from single-trial non-invasive fMRI recordings, in which a three-dimensional convolutional network augmented with information bottleneck is developed to …

abstract arxiv basic brain brain activity challenge continuous cs.cl decoding devices language multiple natural natural language noise signal through type