Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning

arXiv:2310.07518v2 Announce Type: replace
Abstract: Posterior sampling allows exploitation of prior knowledge on the environment's transition dynamics to improve the sample efficiency of reinforcement learning. The prior is typically specified as a class of parametric distributions, the design of which can be cumbersome in practice, often resulting in the choice of uninformative priors. In this work, we propose a novel posterior sampling approach in which the prior is given as a (partial) causal graph over the environment's variables. The latter …

abstract arxiv causal class cs.lg design dynamics efficiency environment exploitation graph knowledge parametric posterior practice prior reinforcement reinforcement learning sample sampling the environment transition type