[D] Relationship Between Neural Networks and Universal Approximation Theorem

Web: https://www.reddit.com/r/MachineLearning/comments/scfb7i/d_relationship_between_neural_networks_and/

https://en.m.wikipedia.org/wiki/Universal_approximation_theorem

For this theorem to work, i.e. for the function fϵfϵ to approximate the function ff at some arbitrary level of precision - this theorem seems to specifically require that the function σσ to not be a polynomial.

In a certain sense, it seems like being polynomial here has a negative connotation, in the sense that it might prevent the results of this theorem from being true.

Regarding this, I had the following questions:

Could this theorem have been written …

machinelearning networks neural neural networks relationship theorem