Refining Minimax Regret for Unsupervised Environment Design

arXiv:2402.12284v1 Announce Type: new
Abstract: In unsupervised environment design, reinforcement learning agents are trained on environment configurations (levels) generated by an adversary that maximises some objective. Regret is a commonly used objective that theoretically results in a minimax regret (MMR) policy with desirable robustness guarantees; in particular, the agent's maximum regret is bounded. However, once the agent reaches this regret bound on all levels, the adversary will only sample levels where regret cannot be further reduced. Although there are possible …

abstract agent agents arxiv cs.ai cs.lg design environment generated minimax policy reinforcement reinforcement learning robustness type unsupervised