FORML: A Riemannian Hessian-free Method for Meta-learning with Orthogonality Constraint

arXiv:2402.18605v1 Announce Type: new
Abstract: Meta-learning problem is usually formulated as a bi-level optimization in which the task-specific and the meta-parameters are updated in the inner and outer loops of optimization, respectively. However, performing the optimization in the Riemannian space, where the parameters and meta-parameters are located on Riemannian manifolds is computationally intensive. Unlike the Euclidean methods, the Riemannian backpropagation needs computing the second-order derivatives that include backward computations through the Riemannian operators such as retraction and orthogonal projection. This …

abstract arxiv cs.lg free meta meta-learning optimization parameters space type