Sigmoid
A sigmoid is a smooth, bounded, S-shaped function that maps real-valued inputs between two limiting values, commonly zero and one.
Grokking Machine Learning
A function that maps a real-valued input to a value between zero and one, commonly interpreted as a binary class probability.
- How it is used
- Maps classifier scores to continuous 0–1 predictions for sentiment classification.
- How deep it goes
- Explains the input-output pattern, midpoint, continuity, differentiability, and contrast with the step function.
- Examples
- Uses sentence-happiness predictions and sample sigmoid outputs for inputs from −5 to 5.
- What you can do after
- Understand how sigmoid scores become continuous class predictions, with 0.5 separating two classes.
Inside Deep Learning: Math, Algorithms, Models
A nonlinear activation function that maps real-valued inputs to a bounded interval.
- How it is used
- Uses sigmoid to add nonlinearity, regulate LSTM information flow, or constrain GAN outputs to [0,1].
- How deep it goes
- Defines the range, formula, placement, and limitations, while deferring fuller activation-function properties.
- Examples
- Demonstrates sigmoid through plots, LSTM gates, and an optional nn.Sigmoid output layer in a GAN generator.
- What you can do after
- Apply it via PyTorch as a hidden activation, an LSTM gate component, or a bounded GAN generator output.
Read how Inside Deep Learning: Math, Algorithms, Models explains sigmoid →
Statistics Every Programmer Needs
A sigmoid function is a function with an S-shaped curve that transitions smoothly between lower and upper limiting values.
- How it is used
- Transforms a linear combination of predictors into a probability for binary classification.
- How deep it goes
- Explains the S-shaped transformation, fitted-coefficient calculation, and classification threshold; does not develop the sigmoid independently of logistic regression.
- Examples
- Calculates raisin-class probabilities from minor-axis length and perimeter, then compares thresholded predictions with recorded classes.
- What you can do after
- Apply it via fitted logistic-regression coefficients to calculate binary-class probabilities and make threshold-based predictions.
Read how Statistics Every Programmer Needs explains sigmoid →
Grokking Statistics
An S-curve is a sigmoid-shaped relationship in which growth is initially rapid, then slows toward a plateau.
- How it is used
- Maps inputs to bounded probabilities for binary classification and frames capability gains against data-collection expense.
- How deep it goes
- Defines the curve, interprets its inflection point, gives its formula, and defers the mathematics of maximum-likelihood fitting.
- Examples
- Uses toddler meltdowns versus hours without a nap, Python plots, and capability coverage that plateaus with increasing expense.
- What you can do after
- Apply it via scikit-learn or the formula to predict a binary outcome’s probability from an input.
Grokking AI Algorithms, Second Edition
A sigmoid function is a bounded S-shaped mathematical function that maps real-valued inputs to values between two limits, commonly zero and one.
- How it is used
- Turns weighted sums into bounded neuron activations and probability-like predictions in perceptrons and artificial neural networks.
- How deep it goes
- Walks through layer-by-layer activation and implements sigmoid and its derivative for backpropagation.
- Examples
- Uses a collision prediction of 0.00214 and a loan-decision network with sigmoid hidden layers and a step-function output.
- What you can do after
- Build it from scratch in NumPy for forward propagation and derivative-based weight updates.
Read how Grokking AI Algorithms, Second Edition explains sigmoid →
Data Without Labels
A sigmoid function is a differentiable S-shaped function that maps real-valued inputs to bounded output values and is commonly used to produce binary-classification scores.
- How it is used
- Turns fitted classification scores into event probabilities and serves as a neural-network activation for binary outputs.
- How deep it goes
- Defines the logistic equation and logit relationship, then explains bounded outputs and activation roles without a training procedure.
- Examples
- Contrasts linear and sigmoid probability curves and uses customer classification to illustrate predicted event probabilities.
- What you can do after
- Evaluate/understand why binary classifiers use a sigmoid to produce bounded probability scores.
Natural Language Processing in Action, Second Edition
A logistic function is an S-shaped mathematical function that maps an input to a bounded continuous value and provides a soft threshold.
- How it is used
- Deploys sigmoid as a soft binary decision in logistic regression, perceptrons, neural networks, and independent multi-label outputs.
- How deep it goes
- Explains the S-shaped behavior, parameter effects, model equivalence, PyTorch implementation, training loss, and multi-label use.
- Examples
- Uses plotted logistic curves, PyTorch name classification, binary cross-entropy, training metrics, and predictions for sample names.
- What you can do after
- Build it from scratch with NumPy or PyTorch, and apply it to binary classification or independent multi-label prediction.
Read how Natural Language Processing in Action, Second Edition explains sigmoid →
Knowledge Graphs and LLMs in Action
A sigmoid activation maps a real-valued score to a value between zero and one, commonly interpreted as a probability for a binary outcome.
- How it is used
- Uses sigmoid to turn classifier or link-compatibility scores into interpretable probabilities for binary prediction.
- How deep it goes
- Explains score-to-probability conversion and loss integration, but does not derive the sigmoid mathematically.
- Examples
- Uses graph-node classification and user–movie recommendation link prediction as concrete applications.
- What you can do after
- Apply it via PyTorch’s F.binary_cross_entropy_with_logits to train probability-producing link-prediction models.
Read how Knowledge Graphs and LLMs in Action explains sigmoid →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A sigmoid is a smooth mathematical function that maps real-valued inputs to values between zero and one.
- How it is used
- Converts logits from classification and deployed model endpoints into probabilities suitable for interpretation.
- How deep it goes
- Explains the formula, smooth bounded behavior, logit relationship, and application to model outputs.
- Examples
- Demonstrates gradient boosting probability histograms and a property prediction yielding 17.9% probability above the median.
- What you can do after
- Apply it via TensorFlow to transform raw model predictions into interpretable probabilities.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains sigmoid →
How Large Language Models Work
An S-curve is a growth pattern in which progress accelerates, reaches a transition region, and then approaches a plateau.
- How it is used
- Frames claims about technological and LLM self-improvement against diminishing returns and practical constraints.
- How deep it goes
- Defines plateau behavior, explains constraints, and contrasts ordinary with iterative S-curves without mathematical derivation.
- Examples
- Uses transistor growth, system performance, GPUs, and successive technological techniques as examples.
- What you can do after
- Evaluate claims of boundless improvement by identifying plateaus, constraints, and possible new techniques.
Learn Generative AI with PyTorch
An activation function that maps outputs into the range 0 to 1, used here to interpret the discriminator output as a probability.
- How it is used
- Uses sigmoid to convert a single neural-network output into a probability for binary classification or discriminator decisions.
- How deep it goes
- Defines the output range and placement in PyTorch models, but does not derive the sigmoid formula or explain its gradients.
- Examples
- Uses ankle-boot classification, t-shirt classification, and real-versus-fake discriminator outputs as examples.
- What you can do after
- Apply it via PyTorch by placing nn.Sigmoid() after a one-unit final linear layer for binary model outputs.
Read how Learn Generative AI with PyTorch explains sigmoid →
Deep Learning with R, Third Edition
A sigmoid activation function maps arbitrary real-valued inputs to values in the interval from zero to one.
- How it is used
- Deploys sigmoid as the final activation for binary classification, producing a scalar interpreted as the positive-class probability.
- How deep it goes
- Defines sigmoid as a squashing function, explains its probability interpretation, and contrasts its bounded range with linear regression outputs.
- Examples
- Uses a sentiment-classification model whose final one-unit Dense layer applies sigmoid.
- What you can do after
- Apply it via Keras as a one-unit final-layer activation for binary classification and pair it with binary crossentropy.
Read how Deep Learning with R, Third Edition explains sigmoid →
Machine Learning Algorithms in Depth
A sigmoid function is a continuous S-shaped function that maps real-valued inputs to values between zero and one.
- How it is used
- Converts real-valued scores into probability-like values for binary variational inference and neural-network classification.
- How deep it goes
- Uses sigmoid outputs in update equations and implementations, but does not develop its mathematical derivation or numerical properties.
- Examples
- Uses binary image denoising and duplicate-question prediction as concrete applications.
- What you can do after
- Apply it via scipy.special.expit or a neural-network sigmoid activation in probabilistic and classification workflows.
Read how Machine Learning Algorithms in Depth explains sigmoid →
Deep Learning with Python, Third Edition
A statistical classification method that models class probabilities with a linear predictor passed through a logistic function.
- How it is used
- Maps a binary classifier’s final numeric output to a probability-like score between 0 and 1.
- How deep it goes
- Defines the output range and placement, but stops short of deriving the sigmoid formula or explaining its internals.
- Examples
- Uses sentiment classification and dogs-versus-cats image classification as concrete binary-output examples.
- What you can do after
- Apply it via Keras by configuring one sigmoid output unit and binary crossentropy for binary classification.
Read how Deep Learning with Python, Third Edition explains sigmoid →
Math and Architectures of Deep Learning
An S-shaped nonlinear activation function whose steepness and horizontal position can be adjusted with parameters.
- How it is used
- Uses a nonlinear activation to extend weighted-sum models toward more complex classification and function-approximation tasks.
- How deep it goes
- Defines the curve, parameter effects, differentiability, and derivative behavior, but does not provide implementation code.
- Examples
- Uses weighted-sum classification, parameterized curves, derivative-overlaid graphs, and transformed manifolds as examples.
- What you can do after
- Evaluate how sigmoid steepness, position, saturation, and differentiability affect neural-network classification and training.
Read how Math and Architectures of Deep Learning explains sigmoid →
Reinforcement Learning from Human Feedback
The sigmoid function maps a real-valued input to a number between zero and one using the function 1/(1+e^-x).
- How it is used
- Converts reward-score differences into probabilities for pairwise preference modeling.
- How deep it goes
- Defines the function and derives its use in equivalent likelihood and negative-log-likelihood objectives.
- Examples
- Uses chosen and rejected language-model completions scored under a prompt.
- What you can do after
- Apply sigmoid within a Bradley–Terry reward-model objective for pairwise preference training.
Read how Reinforcement Learning from Human Feedback explains sigmoid →
Related concepts
- activation functionkind of · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
- logistic regressionused by · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.
- language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- probability theoryproduces · Probability theory is the branch of mathematics that formalizes uncertainty and analyzes random events and outcomes using probability measures.
- cross entropy lossused by · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.