An activation function transforms a neuron's weighted input before producing its output. Its nonlinearity lets stacked neural-network layers learn complex patterns and decision boundaries; without it, multiple layers collapse into one weighted-sum layer. section 4.1.1
Common activation functions serve different modeling needs:
- Sigmoid and TanH squash inputs into bounded ranges, making them useful for binary classification or normalized outputs. section 4.1.1
- ReLU offers faster computation and helps mitigate the vanishing-gradient problem associated with Sigmoid and TanH toward their extreme ranges. section 4.1.1
- The step function provides the perceptron's earlier activation approach, while Sigmoid, TanH, and ReLU are more evolved choices for deep networks. section 4.1.1
Why nonlinearity matters
Consider several hidden layers that each compute weighted sums. If every layer remains linear, multiplying the weight matrices produces an equivalent single layer. Applying an activation function between layers prevents that collapse and enables richer representations for fraud-detection models. section 4.1.1
Figure 4.5 Some examples of activation functions commonly used in deep neural networks – Sigmoid, TanH, and ReLU.
Compare the shapes and ranges of Sigmoid, TanH, and ReLU when selecting an activation function. section 4.1.1



















