Deep Q-learning: a robust control approach. (arXiv:2201.08610v2 [cs.LG] UPDATED)

In this paper, we place deep Q-learning into a control-oriented perspective
and study its learning dynamics with well-established techniques from robust
control. We formulate an uncertain linear time-invariant model by means of the
neural tangent kernel to describe learning. We show the instability of learning
and analyze the agent's behavior in frequency-domain. Then, we ensure
convergence via robust controllers acting as dynamical rewards in the loss
function. We synthesize three controllers: state-feedback gain scheduling H2,
dynamic Hinf, and constant gain …

arxiv control q-learning