Optimization without retraction on the random generalized Stiefel manifold

arXiv:2405.01702v1 Announce Type: new
Abstract: Optimization over the set of matrices that satisfy $X^\top B X = I_p$, referred to as the generalized Stiefel manifold, appears in many applications involving sampled covariance matrices such as canonical correlation analysis (CCA), independent component analysis (ICA), and the generalized eigenvalue problem (GEVP). Solving these problems is typically done by iterative methods, such as Riemannian approaches, which require a computationally expensive eigenvalue decomposition involving fully formed $B$. We propose a cheap stochastic iterative method …

abstract analysis applications arxiv canonical correlation covariance cs.lg eigenvalue generalized independent manifold math.oc optimization random retraction set stat.ml type