Geometric Policy Iteration for Markov Decision Processes. (arXiv:2206.05809v2 [cs.LG] UPDATED)

Recently discovered polyhedral structures of the value function for finite
state-action discounted Markov decision processes (MDP) shed light on
understanding the success of reinforcement learning. We investigate the value
function polytope in greater detail and characterize the polytope boundary
using a hyperplane arrangement. We further show that the value space is a union
of finitely many cells of the same hyperplane arrangement and relate it to the
polytope of the classical linear programming formulation for MDPs. Inspired by
these geometric …

arxiv decision iteration lg markov policy processes