Language Model Training Paradigms for Clinical Feature Embeddings. (arXiv:2311.00768v1 [cs.LG])

In research areas with scarce data, representation learning plays a
significant role. This work aims to enhance representation learning for
clinical time series by deriving universal embeddings for clinical features,
such as heart rate and blood pressure. We use self-supervised training
paradigms for language models to learn high-quality clinical feature
embeddings, achieving a finer granularity than existing time-step and
patient-level representation learning. We visualize the learnt embeddings via
unsupervised dimension reduction techniques and observe a high degree of
consistency with …

arxiv clinical data embeddings feature features language language model language models learn quality rate representation representation learning research role series supervised training time series training work