Model approximation in MDPs with unbounded per-step cost

arXiv:2402.08813v1 Announce Type: cross
Abstract: We consider the problem of designing a control policy for an infinite-horizon discounted cost Markov decision process $\mathcal{M}$ when we only have access to an approximate model $\hat{\mathcal{M}}$. How well does an optimal policy $\hat{\pi}^{\star}$ of the approximate model perform when used in the original model $\mathcal{M}$? We answer this question by bounding a weighted norm of the difference between the value function of $\hat{\pi}^\star $ when used in $\mathcal{M}$ and the optimal value function …

abstract approximation arxiv control cost cs.lg cs.sy decision designing eess.sy horizon markov math.oc per policy process star type