Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks

arXiv:2312.14922v2 Announce Type: replace-cross
Abstract: Neural networks excel at discovering statistical patterns in high-dimensional data sets. In practice, higher-order cumulants, which quantify the non-Gaussian correlations between three or more variables, are particularly important for the performance of neural networks. But how efficient are neural networks at extracting features from higher-order cumulants? We study this question in the spiked cumulant model, where the statistician needs to recover a privileged direction or "spike" from the order-$p\ge 4$ cumulants of $d$-dimensional inputs. We …

abstract arxiv cond-mat.stat-mech correlations cs.lg data data sets excel features hypothesis networks neural networks patterns performance practice random statistical statistics stat.ml tests type variables