Programmatic Reward Design by Example. (arXiv:2112.08438v2 [cs.LG] UPDATED)

Reward design is a fundamental problem in reinforcement learning (RL). A
misspecified or poorly designed reward can result in low sample efficiency and
undesired behaviors. In this paper, we propose the idea of programmatic reward
design, i.e. using programs to specify the reward functions in RL environments.
Programs allow human engineers to express sub-goals and complex task scenarios
in a structured and interpretable way. The challenge of programmatic reward
design, however, is that while humans can provide the high-level structures, …

arxiv design