Partial Information as Full: Reward Imputation with Sketching in Bandits. (arXiv:2210.06719v2 [cs.LG] UPDATED)

We focus on the setting of contextual batched bandit (CBB), where a batch of
rewards is observed from the environment in each episode. But the rewards of
the non-executed actions are unobserved (i.e., partial-information feedbacks).
Existing approaches for CBB usually ignore the rewards of the non-executed
actions, resulting in feedback information being underutilized. In this paper,
we propose an efficient reward imputation approach using sketching for CBB,
which completes the unobserved rewards with the imputed rewards approximating
the full-information feedbacks. …

arxiv imputation information