On Pre-Training for Federated Learning. (arXiv:2206.11488v1 [cs.LG])

In most of the literature on federated learning (FL), neural networks are
initialized with random weights. In this paper, we present an empirical study
on the effect of pre-training on FL. Specifically, we aim to investigate if
pre-training can alleviate the drastic accuracy drop when clients'
decentralized data are non-IID. We focus on FedAvg, the fundamental and most
widely used FL algorithm. We found that pre-training does largely close the gap
between FedAvg and centralized learning under non-IID data, but …

arxiv federated learning learning lg pre-training training