Coordinating Momenta for Cross-silo Federated Learning. (arXiv:2102.03970v2 [cs.LG] UPDATED)

Communication efficiency is crucial for federated learning (FL). Conducting
local training steps in clients to reduce the communication frequency between
clients and the server is a common method to address this issue. However, this
strategy leads to the client drift problem due to \textit{non-i.i.d.} data
distributions in different clients which severely deteriorates the performance.
In this work, we propose a new method to improve the training performance in
cross-silo FL via maintaining double momentum buffers. In our algorithm, one
momentum …

arxiv federated learning learning