SignSGD with Federated Voting

arXiv:2403.16372v1 Announce Type: new
Abstract: Distributed learning is commonly used for accelerating model training by harnessing the computational capabilities of multiple-edge devices. However, in practical applications, the communication delay emerges as a bottleneck due to the substantial information exchange required between workers and a central parameter server. SignSGD with majority voting (signSGD-MV) is an effective distributed learning algorithm that can significantly reduce communication costs by one-bit quantization. However, due to heterogeneous computational capabilities, it fails to converge when the mini-batch …

abstract applications arxiv capabilities communication computational cs.dc cs.lg delay devices distributed distributed learning edge edge devices eess.sp however information multiple practical server training type voting workers