Data-driven emergence of convolutional structure in neural networks. (arXiv:2202.00565v2 [cond-mat.dis-nn] UPDATED)

Exploiting data invariances is crucial for efficient learning in both
artificial and biological neural circuits. Understanding how neural networks
can discover appropriate representations capable of harnessing the underlying
symmetries of their inputs is thus crucial in machine learning and
neuroscience. Convolutional neural networks, for example, were designed to
exploit translation symmetry and their capabilities triggered the first wave of
deep learning successes. However, learning convolutions directly from
translation-invariant data with a fully-connected network has so far proven
elusive. Here, we …

arxiv data data-driven networks neural networks