Multimodal self-supervised learning for lesion localization

arXiv:2401.01524v2 Announce Type: replace
Abstract: Multimodal deep learning utilizing imaging and diagnostic reports has made impressive progress in the field of medical imaging diagnostics, demonstrating a particularly strong capability for auxiliary diagnosis in cases where sufficient annotation information is lacking. Nonetheless, localizing diseases accurately without detailed positional annotations remains a challenge. Although existing methods have attempted to utilize local information to achieve fine-grained semantic alignment, their capability in extracting the fine-grained semantics of the comprehensive context within reports is limited. …

abstract annotation annotations arxiv capability cases challenge cs.cv deep learning diagnosis diagnostic diagnostics diseases imaging information localization medical medical imaging multimodal multimodal deep learning progress reports self-supervised learning supervised learning type