Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation. (arXiv:2203.13410v1 [cs.LG])

In this article, we prove approximation theorems in classes of deep and
shallow neural networks with analytic activation functions by elementary
arguments. We prove for both real and complex networks with non-polynomial
activation that the closure of the class of neural networks coincides with the
closure of the space of polynomials. The closure can further be characterized
by the Stone-Weierstrass theorem (in the real case) and Mergelyan's theorem (in
the complex case). In the real case, we further prove approximation …

arxiv elementary network neural network polynomial