When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning

arXiv:2406.17718v1 Announce Type: new
Abstract: We investigate the impact of auxiliary learning tasks such as observation reconstruction and latent self-prediction on the representation learning problem in reinforcement learning. We also study how they interact with distractions and observation functions in the MDP. We provide a theoretical analysis of the learning dynamics of observation reconstruction, latent self-prediction, and TD learning in the presence of distractions and observation functions under linear model assumptions. With this formalization, we are able to explain why …

abstract analysis arxiv cs.lg distractions functions impact observation prediction problem reinforcement reinforcement learning representation representation learning study tasks type understanding