How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model

arXiv:2404.10727v1 Announce Type: cross
Abstract: Understanding what makes high-dimensional data learnable is a fundamental question in machine learning. On the one hand, it is believed that the success of deep learning lies in its ability to build a hierarchy of representations that become increasingly more abstract with depth, going from simple features like edges to more complex concepts. On the other hand, learning to be insensitive to invariances of the task, such as smooth transformations for image datasets, has been …

abstract arxiv become build cond-mat.dis-nn cs.lg data deep learning hierarchical learn lies machine machine learning networks question random stat.ml success type understanding