Prior-dependent analysis of posterior sampling reinforcement learning with function approximation

arXiv:2403.11175v1 Announce Type: cross
Abstract: This work advances randomized exploration in reinforcement learning (RL) with function approximation modeled by linear mixture MDPs. We establish the first prior-dependent Bayesian regret bound for RL with function approximation; and refine the Bayesian regret analysis for posterior sampling reinforcement learning (PSRL), presenting an upper bound of ${\mathcal{O}}(d\sqrt{H^3 T \log T})$, where $d$ represents the dimensionality of the transition kernel, $H$ the planning horizon, and $T$ the total number of interactions. This signifies a methodological …

abstract advances analysis approximation arxiv bayesian cs.ai cs.it cs.lg exploration function linear math.it math.st posterior presenting prior refine reinforcement reinforcement learning sampling stat.ml stat.th type work