Model-Free Robust $\phi$-Divergence Reinforcement Learning Using Both Offline and Online Data

arXiv:2405.05468v1 Announce Type: new
Abstract: The robust $\phi$-regularized Markov Decision Process (RRMDP) framework focuses on designing control policies that are robust against parameter uncertainties due to mismatches between the simulator (nominal) model and real-world settings. This work makes two important contributions. First, we propose a model-free algorithm called Robust $\phi$-regularized fitted Q-iteration (RPQ) for learning an $\epsilon$-optimal robust policy that uses only the historical data collected by rolling out a behavior policy (with robust exploratory requirement) on the nominal model. …

abstract algorithm arxiv control cs.lg data decision designing divergence framework free markov offline phi policies process reinforcement reinforcement learning robust simulator stat.ml type work world