Efficient Duple Perturbation Robustness in Low-rank MDPs

arXiv:2404.08089v1 Announce Type: new
Abstract: The pursuit of robustness has recently been a popular topic in reinforcement learning (RL) research, yet the existing methods generally suffer from efficiency issues that obstruct their real-world implementation. In this paper, we introduce duple perturbation robustness, i.e. perturbation on both the feature and factor vectors for low-rank Markov decision processes (MDPs), via a novel characterization of $(\xi,\eta)$-ambiguity sets. The novel robust MDP formulation is compatible with the function representation view, and therefore, is naturally …

abstract arxiv cs.lg efficiency feature implementation low math.oc paper popular reinforcement reinforcement learning research robustness type vectors world