Theoretical Hardness and Tractability of POMDPs in RL with Partial Online State Information

arXiv:2306.08762v3 Announce Type: replace
Abstract: Partially observable Markov decision processes (POMDPs) have been widely applied in various real-world applications. However, existing theoretical results have shown that learning in POMDPs is intractable in the worst case, where the main challenge lies in the lack of latent state information. A key fundamental question here is: how much online state information (OSI) is sufficient to achieve tractability? In this paper, we establish a lower bound that reveals a surprising hardness result: unless we …

abstract applications arxiv case challenge cs.ai cs.lg decision however information lies markov observable processes results state type world