An Analysis of Switchback Designs in Reinforcement Learning

arXiv:2403.17285v1 Announce Type: cross
Abstract: This paper offers a detailed investigation of switchback designs in A/B testing, which alternate between baseline and new policies over time. Our aim is to thoroughly evaluate the effects of these designs on the accuracy of their resulting average treatment effect (ATE) estimators. We propose a novel "weak signal analysis" framework, which substantially simplifies the calculations of the mean squared errors (MSEs) of these ATEs in Markov decision process environments. Our findings suggest that (i) …

abstract a/b testing accuracy aim analysis arxiv b testing cs.lg designs effects investigation novel paper policies reinforcement reinforcement learning stat.ml testing treatment type