Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries

arXiv:2403.09940v1 Announce Type: new
Abstract: Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We propose a policy gradient based approach that is robust to adversarial agents which can send arbitrary values to the server. Under this setting, our results form the first global convergence guarantees with general parametrization. These results demonstrate resilience with adversaries, …

abstract adversarial agents arxiv build convergence cs.ai cs.lg decision decision making global gradient however making math.oc multiple policy raw reinforcement reinforcement learning results robust small type