Exploring the Complexity of Deep Neural Networks through Functional Equivalence

arXiv:2305.11417v3 Announce Type: replace
Abstract: We investigate the complexity of deep neural networks through the lens of functional equivalence, which posits that different parameterizations can yield the same network function. Leveraging the equivalence property, we present a novel bound on the covering number for deep neural networks, which reveals that the complexity of neural networks can be reduced. Additionally, we demonstrate that functional equivalence benefits optimization, as overparameterized networks tend to be easier to train since increasing network width leads …

abstract arxiv complexity cs.lg function functional lens math.st network networks neural networks novel property replace stat.th through type