Optimal Baseline Corrections for Off-Policy Contextual Bandits

arXiv:2405.05736v1 Announce Type: new
Abstract: The off-policy learning paradigm allows for recommender systems and general ranking applications to be framed as decision-making problems, where we aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. With unbiasedness comes potentially high variance, and prevalent methods exist to reduce estimation variance. These methods typically make use of control variates, either additive (i.e., baseline corrections or doubly robust methods) or multiplicative (i.e., self-normalisation). Our work unifies these …

abstract aim applications arxiv cs.ir cs.lg decision general learn making offline paradigm policies policy ranking recommender systems reduce systems type unbiased variance