Accounting for the Sequential Nature of States to Learn Features for Reinforcement Learning. (arXiv:2205.06000v1 [cs.LG])

In this work, we investigate the properties of data that cause popular
representation learning approaches to fail. In particular, we find that in
environments where states do not significantly overlap, variational
autoencoders (VAEs) fail to learn useful features. We demonstrate this failure
in a simple gridworld domain, and then provide a solution in the form of metric
learning. However, metric learning requires supervision in the form of a
distance function, which is absent in reinforcement learning. To overcome this,
we …

accounting arxiv features learning reinforcement reinforcement learning