Leveraging (Biased) Information: Multi-armed Bandits with Offline Data

arXiv:2405.02594v1 Announce Type: new
Abstract: We leverage offline data to facilitate online learning in stochastic multi-armed bandits. The probability distributions that govern the offline data and the online rewards can be different. Without any non-trivial upper bound on their difference, we show that no non-anticipatory policy can outperform the UCB policy by (Auer et al. 2002), even in the presence of offline data. In complement, we propose an online policy MIN-UCB, which outperforms UCB when a non-trivial upper bound is …

abstract arxiv cs.lg data difference information multi-armed bandits offline online learning policy probability show stat.ml stochastic type