Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets. (arXiv:2205.10671v2 [cs.LG] UPDATED)

We present a family $\{\hat{\pi}\}_{p\ge 1}$ of pessimistic learning rules
for offline learning of linear contextual bandits, relying on confidence sets
with respect to different $\ell_p$ norms, where $\hat{\pi}_2$ corresponds to
Bellman-consistent pessimism (BCP), while $\hat{\pi}_\infty$ is a novel
generalization of lower confidence bound (LCB) to the linear setting. We show
that the novel $\hat{\pi}_\infty$ learning rule is, in a sense, adaptively
optimal, as it achieves the minimax performance (up to log factors) against all
$\ell_q$-constrained problems, and as such …

arxiv confidence linear offline