Adaptive Compression in Federated Learning via Side Information

arXiv:2306.12625v3 Announce Type: replace
Abstract: The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, state-of-the-art bitrate-accuracy tradeoffs have been achieved using stochastic compression methods -- in which the client $n$ sends a sample from a client-only probability distribution $q_{\phi^{(n)}}$, and the server estimates the mean of the clients' distributions using these samples. However, such methods do not take full advantage of the FL …

abstract accuracy art arxiv client communication compression cost cs.dc cs.lg federated learning information sample scalable server state stat.ml stochastic type updates via