COMPRER: A Multimodal Multi-Objective Pretraining Framework for Enhanced Medical Image Representation

arXiv:2403.09672v1 Announce Type: cross
Abstract: Substantial advances in multi-modal Artificial Intelligence (AI) facilitate the combination of diverse medical modalities to achieve holistic health assessments. We present COMPRER , a novel multi-modal, multi-objective pretraining framework which enhances medical-image representation, diagnostic inferences, and prognosis of diseases. COMPRER employs a multi-objective training framework, where each objective introduces distinct knowledge to the model. This includes a multimodal loss that consolidates information across different imaging modalities; A temporal loss that imparts the ability to discern …

abstract advances artificial artificial intelligence arxiv combination cs.cv cs.lg diagnostic diseases diverse framework health image inferences intelligence medical modal multi-modal multimodal multi-objective novel pretraining representation training type