Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process. (arXiv:2202.10589v5 [stat.ML] UPDATED)

This paper is concerned with constructing a confidence interval for a target
policy's value offline based on a pre-collected observational data in infinite
horizon settings. Most of the existing works assume no unmeasured variables
exist that confound the observed actions. This assumption, however, is likely
to be violated in real applications such as healthcare and technological
industries. In this paper, we show that with some auxiliary variables that
mediate the effect of actions on the system dynamics, the target policy's …

arxiv confidence decision interval markov policy process