Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines. (arXiv:2211.10902v2 [cs.LG] UPDATED)

Natural and formal languages provide an effective mechanism for humans to
specify instructions and reward functions. We investigate how to generate
policies via RL when reward functions are specified in a symbolic language
captured by Reward Machines, an increasingly popular automaton-inspired
structure. We are interested in the case where the mapping of environment state
to a symbolic (here, Reward Machine) vocabulary -- commonly known as the
labelling function -- is uncertain from the perspective of the agent. We
formulate the …

arxiv case case study deep rl machines study