Decentralized Federated Learning: Balancing Communication and Computing Costs. (arXiv:2107.12048v3 [cs.LG] UPDATED)

Decentralized stochastic gradient descent (SGD) is a driving engine for
decentralized federated learning (DFL). The performance of decentralized SGD is
jointly influenced by inter-node communications and local updates. In this
paper, we propose a general DFL framework, which implements both multiple local
updates and multiple inter-node communications periodically, to strike a
balance between communication efficiency and model consensus. It can provide a
general decentralized SGD analytical framework. We establish strong convergence
guarantees for the proposed DFL algorithm without the assumption …

arxiv communication computing federated learning learning