The committee machine: Computational to statistical gaps in learning a two-layers neural network

arXiv:1806.05451v3 Announce Type: replace
Abstract: Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide a rigorous justification of these approaches for a two-layers neural network model called the committee machine. We also introduce a version of the approximate message passing (AMP) algorithm for the committee machine that allows to perform optimal learning …

abstract arxiv computational compute cond-mat.dis-nn cond-mat.stat-mech cs.lg errors layer machine network networks neural network neural networks physics physics.comp-ph statistical stat.ml tools transitions type