Size and depth of monotone neural networks: interpolation and approximation

arXiv:2207.05275v2 Announce Type: replace
Abstract: We study monotone neural networks with threshold gates where all the weights (other than the biases) are non-negative. We focus on the expressive power and efficiency of representation of such networks. Our first result establishes that every monotone function over $[0,1]^d$ can be approximated within arbitrarily small additive error by a depth-4 monotone network. When $d > 3$, we improve upon the previous best-known construction which has depth $d+1$. Our proof goes by solving the …

abstract approximation arxiv biases cs.lg efficiency every focus function gates interpolation math.oc negative networks neural networks power representation stat.ml study threshold type