Set-based value operators for non-stationary Markovian environments. (arXiv:2207.07271v2 [cs.LG] UPDATED)

This paper analyzes finite state Markov Decision Processes (MDPs) with
uncertain parameters in compact sets and re-examines results from robust MDP
via set-based fixed point theory. We generalize the Bellman and policy
evaluation operators to operators that contract on the space of value functions
and denote them as \emph{value operators}. We generalize these value operators
to act on the space of value function sets and denote them as \emph{set-based
value operators}. We prove that these set-based value operators are
contractions …

arxiv environments operators set value