Web: http://arxiv.org/abs/1905.11425

Jan. 27, 2022, 2:11 a.m. | Zaiwei Chen, Sheng Zhang, Thinh T. Doan, John-Paul Clarke, Siva Theja Maguluri

cs.LG updates on arXiv.org arxiv.org

Motivated by applications in reinforcement learning (RL), we study a
nonlinear stochastic approximation (SA) algorithm under Markovian noise, and
establish its finite-sample convergence bounds under various stepsizes.
Specifically, we show that when using constant stepsize (i.e., $\alpha_k\equiv
\alpha$), the algorithm achieves exponential fast convergence to a neighborhood
(with radius $O(\alpha\log(1/\alpha))$) around the desired limit point. When
using diminishing stepsizes with appropriate decay rate, the algorithm
converges with rate $O(\log(k)/k)$. Our proof is based on Lyapunov drift
arguments, and to handle …

analysis applications arxiv learning math reinforcement learning stochastic

More from arxiv.org / cs.LG updates on arXiv.org

Director, Data Engineering and Architecture

@ Chainalysis | California | New York | Washington DC | Remote - USA

Deep Learning Researcher

@ Topaz Labs | Dallas, TX

Sr Data Engineer (Contractor)

@ SADA | US - West

Senior Cloud Database Administrator

@ Findhelp | Remote

Senior Data Analyst

@ System1 | Remote

Speech Machine Learning Research Engineer

@ Samsung Research America | Mountain View, CA