MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure

arXiv:2405.00902v1 Announce Type: new
Abstract: Multi-agent reinforcement learning (MARL) algorithms often struggle to find strategies close to Pareto optimal Nash Equilibrium, owing largely to the lack of efficient exploration. The problem is exacerbated in sparse-reward settings, caused by the larger variance exhibited in policy learning. This paper introduces MESA, a novel meta-exploration method for cooperative multi-agent learning. It learns to explore by first identifying the agents' high-rewarding joint state-action subspace from training tasks and then learning a set of diverse …

abstract agent algorithms arxiv cs.ai cs.lg cs.ma equilibrium exploration mesa meta multi-agent multi-agent learning nash equilibrium paper pareto policy reinforcement reinforcement learning space state strategies struggle through type variance