MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis

arXiv:2403.15585v1 Announce Type: new
Abstract: Chest X-ray images are commonly used for predicting acute and chronic cardiopulmonary conditions, but efforts to integrate them with structured clinical data face challenges due to incomplete electronic health records (EHR). This paper introduces \textbf{MedPromptX}, the first model to integrate multimodal large language models (MLLMs), few-shot prompting (FP) and visual grounding (VG) to combine imagery with EHR data for chest X-ray diagnosis. A pre-trained MLLM is utilized to complement the missing EHR information, providing a …

arxiv cs.ai cs.cv diagnosis multimodal prompting ray type x-ray