Exploiting Reward Shifting in Value-Based Deep RL. (arXiv:2209.07288v1 [cs.LG])

In this work, we study the simple yet universally applicable case of reward
shaping in value-based Deep Reinforcement Learning (DRL). We show that reward
shifting in the form of the linear transformation is equivalent to changing the
initialization of the $Q$-function in function approximation. Based on such an
equivalence, we bring the key insight that a positive reward shifting leads to
conservative exploitation, while a negative reward shifting leads to
curiosity-driven exploration. Accordingly, conservative exploitation improves
offline RL value estimation, …

arxiv deep rl value