Upper Counterfactual Confidence Bounds: a New Optimism Principle for Contextual Bandits

arXiv:2007.07876v4 Announce Type: replace
Abstract: The principle of optimism in the face of uncertainty is one of the most widely used and successful ideas in multi-armed bandits and reinforcement learning. However, existing optimistic algorithms (primarily UCB and its variants) often struggle to deal with general function classes and large context spaces. In this paper, we study general contextual bandits with an offline regression oracle and propose a simple, generic principle to design optimistic algorithms, dubbed "Upper Counterfactual Confidence Bounds" (UCCB). …

abstract algorithms arxiv confidence counterfactual cs.lg deal face function general however ideas math.st multi-armed bandits optimism reinforcement reinforcement learning stat.ml stat.th struggle type uncertainty variants