Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging

arXiv:2401.02565v3 Announce Type: replace-cross
Abstract: In the context of medical artificial intelligence, this study explores the vulnerabilities of the Pathology Language-Image Pretraining (PLIP) model, a Vision Language Foundation model, under targeted attacks. Leveraging the Kather Colon dataset with 7,180 H&E images across nine tissue types, our investigation employs Projected Gradient Descent (PGD) adversarial perturbation attacks to induce misclassifications intentionally. The outcomes reveal a 100% success rate in manipulating PLIP's predictions, underscoring its susceptibility to adversarial perturbations. The qualitative analysis of …

adversarial arxiv cs.cv eess.iv imaging language language model multimodal pathology q-bio.to type vision vision language model