Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning

arXiv:2109.03445v5 Announce Type: replace-cross
Abstract: Ever since its introduction in the classic paper of Robbins and Monro in 1951, Stochastic Approximation (SA) has become a standard tool for finding a solution of an equation of the form $f(\theta) = 0$, when only noisy measurements of $f(\cdot)$ are available. In most situations, \textit{every component} of the putative solution $\theta_t$ is updated at each step $t$. In some applications such as $Q$-learning, a key technique in Reinforcement Learning (RL), \textit{only one component} …

abstract applications approximation arxiv asynchronous become convergence cs.ai cs.lg cs.sy eess.sy equation form introduction math.pr paper reinforcement reinforcement learning solution standard stat.ml stochastic tool type