TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation

arXiv:2302.09832v3 Announce Type: replace
Abstract: In distributed optimization and learning, several machines alternate between local computations in parallel and communication with a distant server. Communication is usually slow and costly and forms the main bottleneck. This is particularly true in federated learning, where a large number of users collaborate toward a global training task. In addition, it is desirable for a robust algorithm to allow for partial participation, since it is often the case that some clients are not able …

abstract arxiv communication compression cs.lg distributed federated learning forms machines math.oc optimization server training true type