Beyond PCA: A Probabilistic Gram-Schmidt Approach to Feature Extraction

Linear feature extraction at the presence of nonlinear dependencies among the data is a fundamental challenge in unsupervised learning. We propose using a probabilistic Gram-Schmidt (GS) type orthogonalization process in order to detect and map out redundant dimensions. Specifically, by applying the GS process over a family of functions which presumably captures the nonlinear dependencies in the data, we construct a series of covariance matrices that can either be used to identify new large-variance directions, or to remove those dependencies …

beyond challenge cs.it cs.lg data dependencies dimensions extraction family feature functions linear map math.it process schmidt type unsupervised unsupervised learning