Meta-Gradients in Non-Stationary Environments. (arXiv:2209.06159v1 [cs.LG])

Meta-gradient methods (Xu et al., 2018; Zahavy et al., 2020) offer a
promising solution to the problem of hyperparameter selection and adaptation in
non-stationary reinforcement learning problems. However, the properties of
meta-gradients in such environments have not been systematically studied. In
this work, we bring new clarity to meta-gradients in non-stationary
environments. Concretely, we ask: (i) how much information should be given to
the learned optimizers, so as to enable faster adaptation and generalization
over a lifetime, (ii) what meta-optimizer …

arxiv environments meta