Improving Intrinsic Exploration by Creating Stationary Objectives

arXiv:2310.18144v4 Announce Type: replace
Abstract: Exploration bonuses in reinforcement learning guide long-horizon exploration by defining custom intrinsic objectives. Several exploration objectives like count-based bonuses, pseudo-counts, and state-entropy maximization are non-stationary and hence are difficult to optimize for the agent. While this issue is generally known, it is usually omitted and solutions remain under-explored. The key contribution of our work lies in transforming the original non-stationary rewards into stationary rewards through an augmented state representation. For this purpose, we introduce the …

abstract agent arxiv count cs.ai cs.lg entropy exploration guide horizon improving intrinsic issue reinforcement reinforcement learning solutions state type