Short blog-post about how ReLUs approximate functions

I just stumbled upon a neat blog-post on how ReLUs approximate non-linear functions.

When talking to others I often feel like the piece-wise linearity (even of composed layers) is not on peoples mind, but I often think about it when imagining the behavior of networks.

behavior blog functions linear mind networks neuralnetworks non-linear think