Learning Value Functions from Undirected State-only Experience. (arXiv:2204.12458v1 [cs.LG])

This paper tackles the problem of learning value functions from undirected
state-only experience (state transitions without action labels i.e. (s,s',r)
tuples). We first theoretically characterize the applicability of Q-learning in
this setting. We show that tabular Q-learning in discrete Markov decision
processes (MDPs) learns the same value function under any arbitrary refinement
of the action space. This theoretical result motivates the design of Latent
Action Q-learning or LAQ, an offline RL method that can learn effective value
functions from state-only …

arxiv experience learning state value