Communication-Efficient Federated Learning via Regularized Sparse Random Networks

arXiv:2309.10834v2 Announce Type: replace
Abstract: This work presents a new method for enhancing communication efficiency in stochastic Federated Learning that trains over-parameterized random networks. In this setting, a binary mask is optimized instead of the model weights, which are kept fixed. The mask characterizes a sparse sub-network that is able to generalize as good as a smaller target network. Importantly, sparse binary masks are exchanged rather than the floating point weights in traditional federated learning, reducing communication cost to at …

abstract arxiv binary communication cs.cv cs.dc cs.ds cs.lg efficiency federated learning network networks random stochastic trains type via work