Preferences Evolve And So Should Your Bandits: Bandits with Evolving States for Online Platforms

arXiv:2307.11655v2 Announce Type: replace
Abstract: We propose a model for learning with bandit feedback while accounting for deterministically evolving and unobservable states that we call \emph{Bandits with Deterministically Evolving States} ($B-DES$). The workhorse applications of our model are learning for recommendation systems and learning for online ads. In both cases, the reward that the algorithm obtains at each round is a function of the short-term reward of the action chosen and how "healthy" the system is (i.e., as measured by …

abstract accounting applications arxiv call cs.ai cs.gt cs.lg des feedback online platforms platforms recommendation recommendation systems systems type