Adaptive Regularization of Representation Rank as an Implicit Constraint of Bellman Equation

arXiv:2404.12754v1 Announce Type: new
Abstract: Representation rank is an important concept for understanding the role of Neural Networks (NNs) in Deep Reinforcement learning (DRL), which measures the expressive capacity of value networks. Existing studies focus on unboundedly maximizing this rank; nevertheless, that approach would introduce overly complex models in the learning, thus undermining performance. Hence, fine-tuning representation rank presents a challenging and crucial optimization problem. To address this issue, we find a guiding principle for adaptive control of the representation …

arxiv cs.ai cs.lg equation regularization representation type