S$^2$AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic

arXiv:2405.00987v1 Announce Type: new
Abstract: Learning expressive stochastic policies instead of deterministic ones has been proposed to achieve better stability, sample complexity, and robustness. Notably, in Maximum Entropy Reinforcement Learning (MaxEnt RL), the policy is modeled as an expressive Energy-Based Model (EBM) over the Q-values. However, this formulation requires the estimation of the entropy of such EBMs, which is an open problem. To address this, previous MaxEnt RL methods either implicitly estimate the entropy, resulting in high computational complexity and …

actor arxiv cs.lg energy reinforcement reinforcement learning type