A sigmoid is a differentiable, S-shaped function. The logistic sigmoid, S(x) = 1/(1 + e^(−x)), maps real-valued inputs to values between 0 and 1. Its output rises as its input rises. section 1.5.1 section 8.2.4
In binary logistic regression, the sigmoid turns a fitted score into a probability for an event. The related equation log(p/(1 − p)) = β₀ + β₁x₁ expresses the logit as an intercept and a feature coefficient; p is the event probability, and p/(1 − p) is its odds. A straight-line fit is unsuitable for this probability because it can return values below 0 or above 1. section 1.5.1
Predicting customer churn
A classifier trained on customer records can produce a probability that a new customer will churn. The sigmoid keeps that score within the probability range; the score can then inform a binary classification decision. section 1.5.1
In a neural network, a sigmoid can serve as an activation function, commonly in the final output layer for binary classification. Its nonlinearity matters: using only linear activations leaves the network output linear. Tanh is presented as a scaled alternative, generally used in hidden layers. section 8.2.4
Figure 1.14 A linear regression model will not be able to do justice (left); hence, we have logistic regression for classification. Linear regression can generate probability scores more than 1 or less than 0 too, which is mathematically incorrect, whereas the sigmoid function generates probability scores between 0 and 1 only (right).
Compare the unbounded straight-line prediction with the sigmoid’s bounded probability curve. section 1.5.1
