Learning Generalized Medical Image Representations through Image-Graph Contrastive Pretraining

arXiv:2405.09594v1 Announce Type: cross
Abstract: Medical image interpretation using deep learning has shown promise but often requires extensive expert-annotated datasets. To reduce this annotation burden, we develop an Image-Graph Contrastive Learning framework that pairs chest X-rays with structured report knowledge graphs automatically extracted from radiology notes. Our approach uniquely encodes the disconnected graph components via a relational graph convolution network and transformer attention. In experiments on the CheXpert dataset, this novel graph encoding strategy enabled the framework to outperform existing …

abstract annotation arxiv cs.cv cs.lg datasets deep learning eess.iv expert framework generalized graph graphs image interpretation knowledge knowledge graphs medical notes pretraining radiology reduce report through type