Probabilistic Actor-Critic: Learning to Explore with PAC-Bayes Uncertainty

We introduce Probabilistic Actor-Critic (PAC), a novel reinforcement learning algorithm with improved continuous control performance thanks to its ability to mitigate the exploration-exploitation trade-off. PAC achieves this by seamlessly integrating stochastic policies and critics, creating a dynamic synergy between the estimation of critic uncertainty and actor training. The key contribution of our PAC algorithm is that it explicitly models and infers epistemic uncertainty in the critic through Probably Approximately Correct-Bayesian (PAC-Bayes) analysis. This incorporation of critic uncertainty enables PAC to …

actor actor-critic algorithm bayes continuous control cs.lg dynamic exploitation exploration explore key novel performance reinforcement reinforcement learning stochastic synergy the key trade trade-off training uncertainty