Grounded Knowledge-Enhanced Medical VLP for Chest X-Ray

arXiv:2404.14750v1 Announce Type: new
Abstract: Medical vision-language pre-training has emerged as a promising approach for learning domain-general representations of medical image and text. Current algorithms that exploit the global and local alignment between medical image and text could however be marred by the redundant information in medical data. To address this issue, we propose a grounded knowledge-enhanced medical vision-language pre-training (GK-MVLP) framework for chest X-ray. In this framework, medical knowledge is grounded to the appropriate anatomical regions by using a …

abstract algorithms alignment arxiv cs.ai cs.cv current data domain exploit general global however image information issue knowledge language medical medical data pre-training ray text training type vision vision-language x-ray