Predictable Reinforcement Learning Dynamics through Entropy Rate Minimization

In Reinforcement Learning (RL), agents have no incentive to exhibit predictable behaviors, and are often pushed (through e.g. policy entropy regularization) to randomize their actions in favor of exploration. From a human perspective, this makes RL agents hard to interpret and predict, and from a safety perspective, even harder to formally verify. We propose a novel method to induce predictable behavior in RL agents, referred to as Predictability-Aware RL (PA-RL), which employs the state sequence entropy rate as a predictability …

agents cs.ai cs.lg cs.sy dynamics eess.sy entropy exploration human perspective policy rate regularization reinforcement reinforcement learning safety through verify