Language Model Training Paradigms for Clinical Feature Embeddings

In research areas with scarce data, representation learning plays a significant role. This work aims to enhance representation learning for clinical time series by deriving universal embeddings for clinical features, such as heart rate and blood pressure. We use self-supervised training paradigms for language models to learn high-quality clinical feature embeddings, achieving a finer granularity than existing time-step and patient-level representation learning. We visualize the learnt embeddings via unsupervised dimension reduction techniques and observe a high degree of consistency with …

clinical cs.cl cs.lg data embeddings feature features language language model language models language model training learn quality rate representation representation learning research role series supervised training time series training work