A General and Efficient Federated Split Learning with Pre-trained Image Transformers for Heterogeneous Data

arXiv:2403.16050v1 Announce Type: new
Abstract: Federated Split Learning (FSL) is a promising distributed learning paradigm in practice, which gathers the strengths of both Federated Learning (FL) and Split Learning (SL) paradigms, to ensure model privacy while diminishing the resource overhead of each client, especially on large transformer models in a resource-constrained environment, e.g., Internet of Things (IoT). However, almost all works merely investigate the performance with simple neural network models in FSL. Despite the minor efforts focusing on incorporating Vision …

abstract arxiv client cs.cv data distributed distributed learning federated learning general image paradigm practice privacy transformers type