Frequency-Aware Masked Autoencoders for Multimodal Pretraining on Biosignals

arXiv:2309.05927v2 Announce Type: replace
Abstract: Leveraging multimodal information from biosignals is vital for building a comprehensive representation of people's physical and mental states. However, multimodal biosignals often exhibit substantial distributional shifts between pretraining and inference datasets, stemming from changes in task specification or variations in modality compositions. To achieve effective pretraining in the presence of potential distributional shifts, we propose a frequency-aware masked autoencoder ($\texttt{bio}$FAME) that learns to parameterize the representation of biosignals in the frequency space. $\texttt{bio}$FAME incorporates a …

arxiv autoencoders cs.ai cs.lg eess.sp multimodal pretraining type