FedLion: Faster Adaptive Federated Optimization with Fewer Communication

arXiv:2402.09941v1 Announce Type: new
Abstract: In Federated Learning (FL), a framework to train machine learning models across distributed data, well-known algorithms like FedAvg tend to have slow convergence rates, resulting in high communication costs during training. To address this challenge, we introduce FedLion, an adaptive federated optimization algorithm that seamlessly incorporates key elements from the recently proposed centralized adaptive algorithm, Lion (Chen et al. 2o23), into the FL framework. Through comprehensive evaluations on two widely adopted FL benchmarks, we demonstrate …

abstract algorithm algorithms arxiv challenge communication convergence costs cs.ai cs.lg data distributed distributed data faster federated learning framework key machine machine learning machine learning models optimization stat.ml train training type