Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stability. (arXiv:2103.12158v4 [cs.LG] UPDATED)

In this paper, for POMDPs, we provide the convergence of a Q learning
algorithm for control policies using a finite history of past observations and
control actions, and, consequentially, we establish near optimality of such
limit Q functions under explicit filter stability conditions. We present
explicit error bounds relating the approximation error to the length of the
finite history window. We establish the convergence of such Q-learning
iterations under mild ergodicity assumptions on the state process during the
exploration phase. …

arxiv convergence memory near q-learning