FedPFT: Federated Proxy Fine-Tuning of Foundation Models

arXiv:2404.11536v2 Announce Type: replace
Abstract: Adapting Foundation Models (FMs) for downstream tasks through Federated Learning (FL) emerges a promising strategy for protecting data privacy and valuable FMs. Existing methods fine-tune FM by allocating sub-FM to clients in FL, however, leading to suboptimal performance due to insufficient tuning and inevitable error accumulations of gradients. In this paper, we propose Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation in downstream tasks through FL by two key modules. First, the sub-FM …

abstract arxiv cs.ai cs.lg data data privacy error federated learning fine-tuning foundation however performance privacy strategy tasks through type