Representation Learning for Online and Offline RL in Low-rank MDPs. (arXiv:2110.04652v3 [cs.LG] UPDATED)

This work studies the question of Representation Learning in RL: how can we
learn a compact low-dimensional representation such that on top of the
representation we can perform RL procedures such as exploration and
exploitation, in a sample efficient manner. We focus on the low-rank Markov
Decision Processes (MDPs) where the transition dynamics correspond to a
low-rank transition matrix. Unlike prior works that assume the representation
is known (e.g., linear MDPs), here we need to learn the representation for the …

arxiv learning rl