ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training

arXiv:2312.13316v2 Announce Type: replace
Abstract: Despite significant advancements in medical vision-language pre-training, existing methods have largely overlooked the inherent entity-specific context within radiology reports and the complex cross-modality contextual relationships between text and images. To close this gap, we propose a novel Entity-centered Context-aware Medical Vision-language Pre-training (ECAMP) framework, which is designed to enable a more entity-centered and context-sensitive interpretation of medical data. Utilizing the recent powerful large language model, we distill entity-centered context from medical reports, which enables ECAMP …

arxiv context cs.cv language medical pre-training training type vision