Advantage-Aware Policy Optimization for Offline Reinforcement Learning

arXiv:2403.07262v1 Announce Type: new
Abstract: Offline Reinforcement Learning (RL) endeavors to leverage offline datasets to craft effective agent policy without online interaction, which imposes proper conservative constraints with the support of behavior policies to tackle the Out-Of-Distribution (OOD) problem. However, existing works often suffer from the constraint conflict issue when offline datasets are collected from multiple behavior policies, i.e., different behavior policies may exhibit inconsistent actions with distinct returns across the state space. To remedy this issue, recent Advantage-Weighted (AW) …

abstract agent arxiv behavior conflict constraints craft cs.ai cs.lg datasets distribution however issue offline optimization policy reinforcement reinforcement learning support type