Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learning

Federated Learning (FL) is a machine learning paradigm that enables clients to jointly train a global model by aggregating the locally trained models without sharing any local training data. In practice, there can often be substantial heterogeneity (e.g., class imbalance) across the local data distributions observed by each of these clients. Under such non-iid data distributions across clients, FL suffers from the 'client-drift' problem where every client drifts to its own local optimum. This results in slower convergence and poor …

class client cs.lg data distillation drift federated learning global machine machine learning paradigm practice train training training data