Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks

Constructing states from sequences of observations is an important component of reinforcement learning agents. One solution for state construction is to use recurrent neural networks. Back-propagation through time (BPTT), and real-time recurrent learning (RTRL) are two popular gradient-based methods for recurrent learning. BPTT requires complete trajectories of observations before it can compute the gradients and is unsuitable for online updates. RTRL can do online updates but scales poorly to large networks. In this paper, we propose two constraints that make …

agents bptt compute construction gradient networks neural networks popular propagation real-time recurrent neural networks reinforcement reinforcement learning scalable solution state through