Revisiting Decentralized ProxSkip: Achieving Linear Speedup

arXiv:2310.07983v2 Announce Type: replace
Abstract: The ProxSkip algorithm for decentralized and federated learning is gaining increasing attention due to its proven benefits in accelerating communication complexity while maintaining robustness against data heterogeneity. However, existing analyses of ProxSkip are limited to the strongly convex setting and do not achieve linear speedup, where convergence performance increases linearly with respect to the number of nodes. So far, questions remain open about how ProxSkip behaves in the non-convex setting and whether linear speedup is …

abstract algorithm arxiv attention benefits communication complexity convergence cs.lg data decentralized federated learning however linear math.oc performance robustness stat.ml type