On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networks

Federated learning (FL) is a widely employed distributed paradigm for collaboratively training machine learning models from multiple clients without sharing local data. In practice, FL encounters challenges in dealing with partial client participation due to the limited bandwidth, intermittent connection and strict synchronized delay. Simultaneously, there exist few theoretical convergence guarantees in this practical setting, especially when associated with the non-convex optimization of neural networks. To bridge this gap, we focus on the training problem of federated averaging (FedAvg) method …

bandwidth challenges client convergence cs.lg data delay distributed federated learning intermittent machine machine learning machine learning models multiple networks neural networks paradigm practice stat.ml training