Multi-modal vision-language model for generalizable annotation-free pathological lesions localization

arXiv:2401.02044v2 Announce Type: replace
Abstract: Defining pathologies automatically from medical images aids the understanding of the emergence and progression of diseases, and such an ability is crucial in clinical diagnostics. However, existing deep learning models heavily rely on expert annotations and lack generalization capabilities in open clinical environments. In this study, we present a generalizable vision-language pre-training model for Annotation-Free pathological lesions Localization (AFLoc). The core strength of AFLoc lies in its extensive multi-level semantic structure-based contrastive learning, which comprehensively …

abstract annotation annotations arxiv capabilities clinical cs.cv deep learning diagnostics diseases emergence environments expert free however images language language model localization medical modal multi-modal study type understanding vision