Learning Adversarial MDPs with Stochastic Hard Constraints

arXiv:2403.03672v1 Announce Type: new
Abstract: We study online learning problems in constrained Markov decision processes (CMDPs) with adversarial losses and stochastic hard constraints. We consider two different scenarios. In the first one, we address general CMDPs, where we design an algorithm that attains sublinear regret and cumulative positive constraints violation. In the second scenario, under the mild assumption that a policy strictly satisfying the constraints exists and is known to the learner, we design an algorithm that achieves sublinear regret …

abstract adversarial algorithm arxiv constraints cs.lg decision design general losses markov online learning positive processes stochastic study type