Performance Improvement Bounds for Lipschitz Configurable Markov Decision Processes

arXiv:2402.13821v1 Announce Type: new
Abstract: Configurable Markov Decision Processes (Conf-MDPs) have recently been introduced as an extension of the traditional Markov Decision Processes (MDPs) to model the real-world scenarios in which there is the possibility to intervene in the environment in order to configure some of its parameters. In this paper, we focus on a particular subclass of Conf-MDP that satisfies regularity conditions, namely Lipschitz continuity. We start by providing a bound on the Wasserstein distance between $\gamma$-discounted stationary distributions …

abstract arxiv cs.lg decision environment extension improvement markov paper parameters performance possibility processes the environment type world