On Pre-Training for Federated Learning. (arXiv:2206.11488v1 [cs.LG])

Web: http://arxiv.org/abs/2206.11488

In most of the literature on federated learning (FL), neural networks are
initialized with random weights. In this paper, we present an empirical study
on the effect of pre-training on FL. Specifically, we aim to investigate if
pre-training can alleviate the drastic accuracy drop when clients'
decentralized data are non-IID. We focus on FedAvg, the fundamental and most
widely used FL algorithm. We found that pre-training does largely close the gap
between FedAvg and centralized learning under non-IID data, but …

arxiv federated learning learning lg on pre-training training