Why are neural networks optimized instead of just optimizing a high dimensional function?

I know that neural networks are universal approximators when given a sufficient number of neurons, but there are other things that can be universal approximators, such as a Taylor series with a high enough order.

So, my question is that, why can we not just optimize some high parameter count (or high dimensional) function instead? I am using a Taylor series just as an example, it can be any type of high dimensional function, and they all can be tuned …

deeplearning function networks neural networks neurons question series taylor things universal