Bayesian Off-Policy Evaluation and Learning for Large Action Spaces

arXiv:2402.14664v1 Announce Type: new
Abstract: In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach designed for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, …

abstract arxiv bayesian correlations cs.ai cs.lg evaluation framework interactive policy presenting sample spaces stat.ml systems through type