Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications

arXiv:2403.18316v1 Announce Type: new
Abstract: Electronic Health Record (EHR) datasets from Intensive Care Units (ICU) contain a diverse set of data modalities. While prior works have successfully leveraged multiple modalities in supervised settings, we apply advanced self-supervised multi-modal contrastive learning techniques to ICU data, specifically focusing on clinical notes and time-series for clinically relevant online prediction tasks. We introduce a loss function Multi-Modal Neighborhood Contrastive Loss (MM-NCL), a soft neighborhood function, and showcase the excellent linear probe and zero-shot performance …

abstract advanced applications apply arxiv clinical cs.lg data datasets diverse ehr electronic electronic health record health modal multi-modal multiple notes prior series set type units