liveBook · Manning

Tanh

As explained in Timeless Algorithms, The Foundational Ideas

The hyperbolic tangent function maps real-valued inputs to values between −1 and 1 and is centered at zero.

An activation function changes the representation passed between neural-network layers. The hyperbolic tangent function, or tanh, maps real inputs to values between −1 and 1 and centers them at zero: an input of zero produces zero, while positive and negative inputs retain their respective signs. section 11.4.1

tanh⁡(z)=ez-e-zez+e-z

This equation defines tanh in terms of exponentials. section 11.4.1

Two hidden-node inputs

For pre-activation values 1.1 and 0.1, tanh produces approximately 0.80 and 0.10. The smaller value stays near zero, unlike the sigmoid output of about 0.52; the choice of activation therefore changes the representation received by the next layer. section 11.4.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines tanh, explains its range and zero-centered behavior, and compares outputs on two inputs.
Examples
Calculates tanh for pre-activation values 1.1 and 0.1 alongside sigmoid and ReLU outputs.
What you can do after
Evaluate how tanh changes hidden-layer values relative to sigmoid and ReLU.
How it connects
Compares tanh with sigmoid, ReLU, activation functions, and neural-network representations.

Also explained in

Compare how 5 books explain tanh →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.