FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models

arXiv:2403.09904v1 Announce Type: new
Abstract: Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy. A critical bottleneck in FL is the communication cost. A pivotal strategy to mitigate this burden is \emph{Local Training}, which involves running multiple local stochastic gradient descent iterations between communication phases. Our work is inspired by the innovative \emph{Scaffnew} algorithm, which has …

abstract arxiv attention communication cost cs.ai cs.dc cs.lg data distributed federated learning pivotal privacy private data process server strategy training type