MedRG: Medical Report Grounding with Multi-modal Large Language Model

arXiv:2404.06798v1 Announce Type: new
Abstract: Medical Report Grounding is pivotal in identifying the most relevant regions in medical images based on a given phrase query, a critical aspect in medical image analysis and radiological diagnosis. However, prevailing visual grounding approaches necessitate the manual extraction of key phrases from medical reports, imposing substantial burdens on both system efficiency and physicians. In this paper, we introduce a novel framework, Medical Report Grounding (MedRG), an end-to-end solution for utilizing a multi-modal Large Language …

abstract analysis arxiv cs.cv diagnosis extraction however image images key language language model large language large language model medical modal multi-modal pivotal query report reports type visual