ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularization

arXiv:2402.14528v1 Announce Type: new
Abstract: The varying significance of distinct primitive behaviors during the policy learning process has been overlooked by prior model-free RL algorithms. Leveraging this insight, we explore the causal relationship between different action dimensions and rewards to evaluate the significance of various primitive behaviors during training. We introduce a causality-aware entropy term that effectively identifies and prioritizes actions with high potential impacts for efficient exploration. Furthermore, to prevent excessive focus on specific primitive behaviors, we analyze the …

actor actor-critic arxiv causality cs.ai cs.lg entropy policy regularization type