How deeply does Grokking Machine Learning cover tanh?

Explains tanh’s output range, shape, sigmoid comparison, and vanishing-gradient limitation without deriving its formula.

Tanh

As explained in Grokking Machine Learning

A smooth S-shaped activation function that maps real-valued inputs to values between negative one and one.

tanh, or the hyperbolic tangent, is a smooth S-shaped activation function that squishes any real-valued input into the interval from −1 to 1. In a neural network, it helps transform layer outputs so the model can build more complex decision boundaries. section 10.2 section 10.6

tanh is useful, but it has important trade-offs:

It is less flat than sigmoid, so it often works better in practice. section 10.2
It still has flat regions and can contribute to the vanishing-gradient problem during training. section 10.2 section 10.6
It contrasts with ReLU, which turns negative inputs into zero and is used more widely to reduce vanishing gradients. section 10.2

Compare activation-function shapes

Visualize sigmoid, tanh, and ReLU side by side: tanh occupies the middle position, showing its S-shaped curve and output range between −1 and 1.

How Grokking Machine Learning covers it

liveBook manning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Optimization Algorithms: AI techniques for design, planning, and control problemsThe hyperbolic tangent is an activation function that maps real-valued inputs to the interval from -1 to 1.
liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA nonlinear activation function that maps real-valued inputs into the interval [-1,1].
liveBook manning.com

Math and Architectures of Deep LearningA hyperbolic tangent activation function used to introduce nonlinearity in neural networks.

Compare how 4 books explain tanh →

Related concepts

activation functionkind of · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
ReLUcontrasts with · ReLU, or rectified linear unit, is an activation function that maps an input x to max(0, x).
recurrent neural networkused by · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
sigmoidcontrasts with · A sigmoid is a smooth, bounded, S-shaped function that maps real-valued inputs between two limiting values, commonly zero and one.
additive scoreused by · An additive score is a learned attention compatibility function that combines representations to score a query-key pair.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.