Global and Local Prompts Cooperation via Optimal Transport for Federated Learning

arXiv:2403.00041v1 Announce Type: new
Abstract: Prompt learning in pretrained visual-language models has shown remarkable flexibility across various downstream tasks. Leveraging its inherent lightweight nature, recent research attempted to integrate the powerful pretrained models into federated learning frameworks to simultaneously reduce communication costs and promote local training on insufficient data. Despite these efforts, current federated prompt learning methods lack specialized designs to systematically address severe data heterogeneities, e.g., data distribution with both label and feature shifts involved. To address this challenge, …

abstract arxiv communication costs cs.ai cs.dc cs.lg data federated learning flexibility frameworks global language language models nature pretrained models promote prompt prompt learning prompts reduce research tasks training transport type via visual