An Investigation of the Bias-Variance Tradeoff in Meta-Gradients. (arXiv:2209.11303v1 [cs.LG])

Meta-gradients provide a general approach for optimizing the meta-parameters
of reinforcement learning (RL) algorithms. Estimation of meta-gradients is
central to the performance of these meta-algorithms, and has been studied in
the setting of MAML-style short-horizon meta-RL problems. In this context,
prior work has investigated the estimation of the Hessian of the RL objective,
as well as tackling the problem of credit assignment to pre-adaptation behavior
by making a sampling correction. However, we show that Hessian estimation,
implemented for example by …

arxiv bias bias-variance investigation meta variance