Variance Reduction based Experience Replay for Policy Optimization. (arXiv:2208.12341v2 [stat.ML] UPDATED)

For reinforcement learning on complex stochastic systems where many factors
dynamically impact the output trajectories, it is desirable to effectively
leverage the information from historical samples collected in previous
iterations to accelerate policy optimization. Classical experience replay
allows agents to remember by reusing historical observations. However, the
uniform reuse strategy that treats all observations equally overlooks the
relative importance of different samples. To overcome this limitation, we
propose a general variance reduction based experience replay (VRER) framework
that can selectively …

arxiv experience optimization policy variance