C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory

arXiv:2402.16349v1 Announce Type: new
Abstract: Generative Adversarial Imitation Learning (GAIL) trains a generative policy to mimic a demonstrator. It uses on-policy Reinforcement Learning (RL) to optimize a reward signal derived from a GAN-like discriminator. A major drawback of GAIL is its training instability - it inherits the complex training dynamics of GANs, and the distribution shift introduced by RL. This can cause oscillations during training, harming its sample efficiency and final policy performance. Recent work has shown that control theory …

abstract adversarial arxiv control cs.lg cs.sy dynamics eess.sy gan generative imitation learning major policy reinforcement reinforcement learning signal theory training trains type