Prompt-based Personalized Federated Learning for Medical Visual Question Answering

arXiv:2402.09677v1 Announce Type: new
Abstract: We present a novel prompt-based personalized federated learning (pFL) method to address data heterogeneity and privacy concerns in traditional medical visual question answering (VQA) methods. Specifically, we regard medical datasets from different organs as clients and use pFL to train personalized transformer-based VQA models for each client. To address the high computational complexity of client-to-client communication in previous pFL methods, we propose a succinct information sharing system by introducing prompts that are small learnable parameters. …

abstract arxiv concerns cs.cv data datasets federated learning medical novel personalized privacy prompt question question answering regard train transformer type visual