A Reduction-based Framework for Sequential Decision Making with Delayed Feedback

arXiv:2302.01477v5 Announce Type: replace
Abstract: We study stochastic delayed feedback in general multi-agent sequential decision making, which includes bandits, single-agent Markov decision processes (MDPs), and Markov games (MGs). We propose a novel reduction-based framework, which turns any multi-batched algorithm for sequential decision making with instantaneous feedback into a sample-efficient algorithm that can handle stochastic delays in sequential decision making. By plugging different multi-batched algorithms into our framework, we provide several examples demonstrating that our framework not only matches or improves …

abstract agent algorithm arxiv cs.lg decision decision making feedback framework games general making markov multi-agent novel processes sample stochastic study type