[R] Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networks

**Paper**: [https://arxiv.org/abs/2402.09092](https://arxiv.org/abs/2402.09092)

**Abstract**:

>Neural networks have proven to be a highly effective tool for solving complex problems in many areas of life. Recently, their importance and practical usability have further been reinforced with the advent of deep learning. One of the important conditions for the success of neural networks is the choice of an appropriate activation function introducing non-linearity into the model. Many types of these functions have been proposed in the literature in the past, but there is no …

abstract deep learning function importance life machinelearning networks neural networks practical success tool usability