Policy Gradient Methods in the Presence of Symmetries and State Abstractions

arXiv:2305.05666v2 Announce Type: replace
Abstract: Reinforcement learning (RL) on high-dimensional and complex problems relies on abstraction for improved efficiency and generalization. In this paper, we study abstraction in the continuous-control setting, and extend the definition of Markov decision process (MDP) homomorphisms to the setting of continuous state and action spaces. We derive a policy gradient theorem on the abstract MDP for both stochastic and deterministic policies. Our policy gradient results allow for leveraging approximate symmetries of the environment for policy …

abstract abstraction abstractions arxiv continuous control cs.ai cs.lg decision definition efficiency gradient markov paper policy process reinforcement reinforcement learning state study type