Sigmoid

As explained in Statistics Every Programmer Needs

A sigmoid function is a function with an S-shaped curve that transitions smoothly between lower and upper limiting values.

The sigmoid, also called the logistic function, turns a numeric input into a value between 0 and 1. Its S-shaped curve makes it useful in logistic regression: a linear combination of predictors becomes an estimated probability that a binary outcome equals 1. Unlike linear regression, the resulting prediction is a class probability rather than an unrestricted numeric outcome. section 5.1

Figure 5.1 A graphical illustration of the sigmoid function. The sigmoid function is characterized by its S-shaped curve, compared to a linear function, which is characterized by a straight line.

The plotted S-shaped curve shows how the sigmoid differs from the straight line associated with a linear function. section 5.1

Write the sigmoid as p = 1 / (1 + e^(−z)), where z is an intercept plus predictor values multiplied by their fitted coefficients. In a binary model, p represents P(Y = 1 | X). A common decision rule predicts class 1 when p is at least 0.5 and class 0 otherwise. The probability and the final class prediction are distinct outputs: the threshold supplies the decision rule. section 5.1 section 5.2.3

Predicting a raisin class

A fitted model uses raisin measurements to estimate class probability. With its rounded coefficients, the calculation is `z = -11.8048 - 0.0244 * MinorAxisLength + 0.0159 * Perimeter`, followed by `p = 1 / (1 + 2.72**(-z))`. For one raisin, the displayed probability is 0.698821, so the 0.5 rule predicts class 1; its recorded class is 0. That mismatch is a reminder to check predictions against observed outcomes rather than treating a probability above the threshold as proof. section 5.2.3

The sigmoid supplies the probability transformation, not evidence that the fitted model is reliable. Assess logistic regression with model-fit and predictor-significance measures, then examine classification errors and metrics such as accuracy and area under the receiver operating characteristic curve. Exploring the predictors and checking issues such as multicollinearity also matter when selecting a model. section 5.2.2 section 5.3

How Statistics Every Programmer Needs covers it

How deep it goes

Explains the S-shaped transformation, fitted-coefficient calculation, and classification threshold; does not develop the sigmoid independently of logistic regression.

Examples

Calculates raisin-class probabilities from minor-axis length and perimeter, then compares thresholded predictions with recorded classes.

What it assumes

Builds on binary outcomes, predictors, coefficients, and probability while explaining how they enter the sigmoid calculation.

What you can do after

Apply it via fitted logistic-regression coefficients to calculate binary-class probabilities and make threshold-based predictions.

How it connects

Connects logistic regression and linear regression to binary classification, predictor coefficients, and classification metrics.

liveBookmanning.com

Read Statistics Every Programmer Needs on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA function that maps a real-valued input to a value between zero and one, commonly interpreted as a binary class probability.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA nonlinear activation function that maps real-valued inputs to a bounded interval.
• 

  liveBookmanning.com
  Grokking StatisticsAn S-curve is a sigmoid-shaped relationship in which growth is initially rapid, then slows toward a plateau.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA sigmoid function is a bounded S-shaped mathematical function that maps real-valued inputs to values between two limits, commonly zero and one.
• 

  liveBookmanning.com
  Data Without LabelsA sigmoid function is a differentiable S-shaped function that maps real-valued inputs to bounded output values and is commonly used to produce binary-classification scores.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA logistic function is an S-shaped mathematical function that maps an input to a bounded continuous value and provides a soft threshold.
• 

  liveBookmanning.com
  Knowledge Graphs and LLMs in ActionA sigmoid activation maps a real-valued score to a value between zero and one, commonly interpreted as a probability for a binary outcome.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA sigmoid is a smooth mathematical function that maps real-valued inputs to values between zero and one.
• 

  liveBookmanning.com
  How Large Language Models WorkAn S-curve is a growth pattern in which progress accelerates, reaches a transition region, and then approaches a plateau.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchAn activation function that maps outputs into the range 0 to 1, used here to interpret the discriminator output as a probability.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA sigmoid activation function maps arbitrary real-valued inputs to values in the interval from zero to one.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA sigmoid function is a continuous S-shaped function that maps real-valued inputs to values between zero and one.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA statistical classification method that models class probabilities with a linear predictor passed through a logistic function.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningAn S-shaped nonlinear activation function whose steepness and horizontal position can be adjusted with parameters.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackThe sigmoid function maps a real-valued input to a number between zero and one using the function 1/(1+e^-x).

Compare how 16 books explain sigmoid →

Related concepts

• activation functionkind of · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
• logistic regressionused by · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.
• language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• cross entropy lossused by · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
• probability theoryproduces · Probability theory is the branch of mathematics that formalizes uncertainty and analyzes random events and outcomes using probability measures.

This explainer was generated by Manning's knowledge graph from the text of Statistics Every Programmer Needs. The section links point to the parts of the book it draws on.