Online Reinforcement Learning in Non-Stationary Context-Driven Environments

We study online reinforcement learning (RL) in non-stationary environments, where a time-varying exogenous context process affects the environment dynamics. Online RL is challenging in such environments due to "catastrophic forgetting" (CF). The agent tends to forget prior knowledge as it trains on new experiences. Prior approaches to mitigate this issue assume task labels (which are often not available in practice) or use off-policy methods that suffer from instability and poor performance.
We present Locally Constrained Policy Optimization (LCPO), an online …

agent catastrophic forgetting context cs.ai cs.lg dynamics environment environments exogenous issue knowledge labels online reinforcement learning prior process reinforcement reinforcement learning study the environment trains