Policy Optimization with Smooth Guidance Learned from State-Only Demonstrations

arXiv:2401.00162v2 Announce Type: replace
Abstract: The sparsity of reward feedback remains a challenging problem in online deep reinforcement learning (DRL). Previous approaches have utilized offline demonstrations to achieve impressive results in multiple hard tasks. However, these approaches place high demands on demonstration quality, and obtaining expert-like actions is often costly and unrealistic. To tackle these problems, we propose a simple and efficient algorithm called Policy Optimization with Smooth Guidance (POSG), which leverages a small set of state-only demonstrations (where only …

abstract arxiv cs.lg expert feedback guidance however multiple offline optimization policy quality reinforcement reinforcement learning results sparsity state tasks type