Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory

arXiv:2006.04761v2 Announce Type: replace
Abstract: Temporal-difference and Q-learning play a key role in deep reinforcement learning, where they are empowered by expressive nonlinear function approximators such as neural networks. At the core of their empirical successes is the learned feature representation, which embeds rich observations, e.g., images and texts, into the latent space that encodes semantic structures. Meanwhile, the evolution of such a feature representation is crucial to the convergence of temporal-difference and Q-learning.
In particular, temporal-difference learning converges when …

abstract arxiv core cs.lg difference feature function images key learn math.oc mean networks neural networks q-learning reinforcement reinforcement learning representation role stat.ml temporal temporal-difference theory type