The Curse of Diversity in Ensemble-Based Exploration

arXiv:2405.04342v1 Announce Type: new
Abstract: We uncover a surprising phenomenon in deep reinforcement learning: training a diverse ensemble of data-sharing agents -- a well-established exploration strategy -- can significantly impair the performance of the individual ensemble members when compared to standard single-agent training. Through careful analysis, we attribute the degradation in performance to the low proportion of self-generated data in the shared training data for each ensemble member, as well as the inefficiency of the individual ensemble members to learn …

abstract agent agents analysis arxiv cs.lg data diverse diversity ensemble exploration performance reinforcement reinforcement learning standard strategy through training type