Truncated Emphatic Temporal Difference Methods for Prediction and Control. (arXiv:2108.05338v2 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2108.05338

Emphatic Temporal Difference (TD) methods are a class of off-policy
Reinforcement Learning (RL) methods involving the use of followon traces.
Despite the theoretical success of emphatic TD methods in addressing the
notorious deadly triad of off-policy RL, there are still two open problems.
First, followon traces typically suffer from large variance, making them hard
to use in practice. Second, though Yu (2015) confirms the asymptotic
convergence of some emphatic TD methods for prediction problems, there is still
no finite sample …

arxiv difference prediction