Future Prediction Can be a Strong Evidence of Good History Representation in Partially Observable Environments

Learning a good history representation is one of the core challenges of reinforcement learning (RL) in partially observable environments. Recent works have shown the advantages of various auxiliary tasks for facilitating representation learning. However, the effectiveness of such auxiliary tasks has not been fully convincing, especially in partially observable environments that require long-term memorization and inference. In this empirical study, we investigate the effectiveness of future prediction for learning the representations of histories, possibly of extensive length, in partially observable …

advantages challenges core cs.ai cs.lg environments evidence future good history observable prediction reinforcement reinforcement learning representation representation learning tasks