Solution space and storage capacity of fully connected two-layer neural networks with generic activation functions

arXiv:2404.13404v1 Announce Type: cross
Abstract: The storage capacity of a binary classification model is the maximum number of random input-output pairs per parameter that the model can learn. It is one of the indicators of the expressive power of machine learning models and is important for comparing the performance of various models. In this study, we analyze the structure of the solution space and the storage capacity of fully connected two-layer neural networks with general activation functions using the replica …

abstract arxiv binary capacity classification classification model cond-mat.dis-nn cs.lg functions input-output layer learn machine machine learning machine learning models maximum networks neural networks per power random solution space stat.ml storage type