An activation function changes the representation passed between neural-network layers. The hyperbolic tangent function, or tanh, maps real inputs to values between −1 and 1 and centers them at zero: an input of zero produces zero, while positive and negative inputs retain their respective signs. section 11.4.1
This equation defines tanh in terms of exponentials. section 11.4.1
Two hidden-node inputs
For pre-activation values 1.1 and 0.1, tanh produces approximately 0.80 and 0.10. The smaller value stays near zero, unlike the sigmoid output of about 0.52; the choice of activation therefore changes the representation received by the next layer. section 11.4.1




