Easy as ABCs: Unifying Boltzmann Q-Learning and Counterfactual Regret Minimization

arXiv:2402.11835v1 Announce Type: new
Abstract: We propose ABCs (Adaptive Branching through Child stationarity), a best-of-both-worlds algorithm combining Boltzmann Q-learning (BQL), a classic reinforcement learning algorithm for single-agent domains, and counterfactual regret minimization (CFR), a central algorithm for learning in multi-agent domains. ABCs adaptively chooses what fraction of the environment to explore each iteration by measuring the stationarity of the environment's reward and transition dynamics. In Markov decision processes, ABCs converges to the optimal policy with at most an O(A) factor …

abstract agent algorithm arxiv boltzmann child counterfactual cs.gt cs.lg cs.ma domains easy environment multi-agent q-learning reinforcement reinforcement learning the environment through type