Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay

arXiv:2404.10662v1 Announce Type: new
Abstract: We study continual offline reinforcement learning, a practical paradigm that facilitates forward transfer and mitigates catastrophic forgetting to tackle sequential offline tasks. We propose a dual generative replay framework that retains previous knowledge by concurrent replay of generated pseudo-data. First, we decouple the continual learning policy into a diffusion-based generative behavior model and a multi-head action evaluation model, allowing the policy to inherit distributional expressivity for encompassing a progressive range of diverse behaviors. Second, we …

arxiv continual cs.ai cs.lg diffusion generative offline reinforcement reinforcement learning type via