Cliff Diving: Exploring Reward Surfaces in Reinforcement Learning Environments. (arXiv:2205.07015v3 [cs.LG] UPDATED)

Visualizing optimization landscapes has led to many fundamental insights in
numeric optimization, and novel improvements to optimization techniques.
However, visualizations of the objective that reinforcement learning optimizes
(the "reward surface") have only ever been generated for a small number of
narrow contexts. This work presents reward surfaces and related visualizations
of 27 of the most widely used reinforcement learning environments in Gym for
the first time. We also explore reward surfaces in the policy gradient
direction and show for the …

arxiv environments reinforcement reinforcement learning