Extremum-Seeking Action Selection for Accelerating Policy Optimization

arXiv:2404.01598v1 Announce Type: new
Abstract: Reinforcement learning for control over continuous spaces typically uses high-entropy stochastic policies, such as Gaussian distributions, for local exploration and estimating policy gradient to optimize performance. Many robotic control problems deal with complex unstable dynamics, where applying actions that are off the feasible control manifolds can quickly lead to undesirable divergence. In such cases, most samples taken from the ambient action space generate low-value trajectories that hardly contribute to policy improvement, resulting in slow or …

abstract arxiv continuous control cs.ai cs.lg cs.ro deal dynamics entropy exploration gradient optimization performance policies policy reinforcement reinforcement learning robotic spaces stochastic type