MEDBind: Unifying Language and Multimodal Medical Data Embeddings

arXiv:2403.12894v1 Announce Type: new
Abstract: Medical vision-language pretraining models (VLPM) have achieved remarkable progress in fusing chest X-rays (CXR) with clinical texts, introducing image-text data binding approaches that enable zero-shot learning and downstream clinical tasks. However, the current landscape lacks the holistic integration of additional medical modalities, such as electrocardiograms (ECG). We present MEDBind (Medical Electronic patient recorD), which learns joint embeddings across CXR, ECG, and medical text. Using text data as the central anchor, MEDBind features tri-modality binding, delivering …

abstract arxiv clinical cs.cv current data embeddings however image integration landscape language medical medical data multimodal pretraining progress tasks text type vision zero-shot