VDSC: Enhancing Exploration Timing with Value Discrepancy and State Counts

arXiv:2403.17542v1 Announce Type: new
Abstract: Despite the considerable attention given to the questions of \textit{how much} and \textit{how to} explore in deep reinforcement learning, the investigation into \textit{when} to explore remains relatively less researched. While more sophisticated exploration strategies can excel in specific, often sparse reward environments, existing simpler approaches, such as $\epsilon$-greedy, persist in outperforming them across a broader spectrum of domains. The appeal of these simpler strategies lies in their ease of implementation and generality across a wide …

abstract arxiv attention cs.ai cs.lg environments excel exploration explore investigation questions reinforcement reinforcement learning state strategies type value