Federated Q-Learning: Linear Regret Speedup with Low Communication Cost

arXiv:2312.15023v2 Announce Type: replace
Abstract: In this paper, we consider federated reinforcement learning for tabular episodic Markov Decision Processes (MDP) where, under the coordination of a central server, multiple agents collaboratively explore the environment and learn an optimal policy without sharing their raw data. While linear speedup in the number of agents has been achieved for some metrics, such as convergence rate and sample complexity, in similar settings, it is unclear whether it is possible to design a model-free algorithm …

abstract agents arxiv communication cost cs.lg data decision environment explore learn linear low markov multiple paper policy processes q-learning raw raw data reinforcement reinforcement learning server stat.ml tabular the environment type while