Compressed Federated Reinforcement Learning with a Generative Model

arXiv:2404.10635v1 Announce Type: cross
Abstract: Reinforcement learning has recently gained unprecedented popularity, yet it still grapples with sample inefficiency. Addressing this challenge, federated reinforcement learning (FedRL) has emerged, wherein agents collaboratively learn a single policy by aggregating local estimations. However, this aggregation step incurs significant communication costs. In this paper, we propose CompFedRL, a communication-efficient FedRL approach incorporating both \textit{periodic aggregation} and (direct/error-feedback) compression mechanisms. Specifically, we consider compressed federated $Q$-learning with a generative model setup, where a central server …

abstract agents aggregation arxiv challenge communication costs cs.dc cs.lg cs.ma estimations generative however learn paper policy reinforcement reinforcement learning sample type