Sigmoid

As explained in Grokking Machine Learning

A function that maps a real-valued input to a value between zero and one, commonly interpreted as a binary class probability.

How happy is a sentence, rather than just happy or sad? A sigmoid turns a classifier’s real-valued score into a value between 0 and 1. Negative scores map below 0.5, zero maps to 0.5, and positive scores map above 0.5. The function is continuous and differentiable everywhere. section 6.1

From score to value

For inputs −5, 0, and 5, the sigmoid outputs approximately 0.007, 0.5, and 0.993. In a sentence-happiness classifier, these values represent predictions from very unlikely to very likely to be happy. section 6.1

Figure 6.3 Left: The step function used to build discrete perceptrons. It outputs a value of 0 for any negative input and a value of 1 for any input that is positive or zero. It has a discontinuity at zero. Right: The sigmoid function used to build continuous perceptrons. It outputs values less than 0.5 for negative inputs and values greater than 0.5 for positive inputs. At zero, it outputs 0.5. It is continuous and differentiable everywhere.

Compare the sigmoid’s smooth transition with the step function’s abrupt switch between 0 and 1. section 6.1

How Grokking Machine Learning covers it

How deep it goes

Explains the input-output pattern, midpoint, continuity, differentiability, and contrast with the step function.

Examples

Uses sentence-happiness predictions and sample sigmoid outputs for inputs from −5 to 5.

What it assumes

Builds on perceptron scoring and the step function; familiarity with the perceptron algorithm is helpful.

What you can do after

Understand how sigmoid scores become continuous class predictions, with 0.5 separating two classes.

How it connects

Connects sigmoid to perceptrons, logistic regression, and the step function.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA nonlinear activation function that maps real-valued inputs to a bounded interval.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsA sigmoid function is a function with an S-shaped curve that transitions smoothly between lower and upper limiting values.
• 

  liveBookmanning.com
  Grokking StatisticsAn S-curve is a sigmoid-shaped relationship in which growth is initially rapid, then slows toward a plateau.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA sigmoid function is a bounded S-shaped mathematical function that maps real-valued inputs to values between two limits, commonly zero and one.
• 

  liveBookmanning.com
  Data Without LabelsA sigmoid function is a differentiable S-shaped function that maps real-valued inputs to bounded output values and is commonly used to produce binary-classification scores.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA logistic function is an S-shaped mathematical function that maps an input to a bounded continuous value and provides a soft threshold.
• 

  liveBookmanning.com
  Knowledge Graphs and LLMs in ActionA sigmoid activation maps a real-valued score to a value between zero and one, commonly interpreted as a probability for a binary outcome.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA sigmoid is a smooth mathematical function that maps real-valued inputs to values between zero and one.
• 

  liveBookmanning.com
  How Large Language Models WorkAn S-curve is a growth pattern in which progress accelerates, reaches a transition region, and then approaches a plateau.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchAn activation function that maps outputs into the range 0 to 1, used here to interpret the discriminator output as a probability.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA sigmoid activation function maps arbitrary real-valued inputs to values in the interval from zero to one.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA sigmoid function is a continuous S-shaped function that maps real-valued inputs to values between zero and one.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA statistical classification method that models class probabilities with a linear predictor passed through a logistic function.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningAn S-shaped nonlinear activation function whose steepness and horizontal position can be adjusted with parameters.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackThe sigmoid function maps a real-valued input to a number between zero and one using the function 1/(1+e^-x).

Compare how 16 books explain sigmoid →

Related concepts

• activation functionkind of · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
• logistic regressionused by · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.
• language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• probability theoryproduces · Probability theory is the branch of mathematics that formalizes uncertainty and analyzes random events and outcomes using probability measures.
• cross entropy lossused by · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.