liveBook · Manning

Log probability

Also called logprobs

A log probability is the logarithm, usually the natural logarithm, of a probability assigned to an event or outcome.

How 10 Manning books explain it

Build a Reasoning Model (From Scratch)

The logarithm of a model-assigned probability, used here to score how likely a generated token or whole sequence is under the current parameters.

How it is used
Scores and ranks candidate responses during self-refinement, self-consistency tie-breaking, rollout evaluation, and reinforcement-learning updates.
How deep it goes
Derives log-softmax indexing, sequence summation, answer-only averaging, and policy-loss integration, but does not develop broader probability theory.
Examples
Uses Berlin-versus-Bridge answers, seven-token answer averaging, rollout comparisons, and a seven-token synthetic vocabulary.
What you can do after
Apply it via PyTorch to compute token, average answer, and sequence log probabilities for scoring responses and building GRPO losses.

Read how Build a Reasoning Model (From Scratch) explains log probability →

Applied Reinforcement Learning

The logarithm of the probability or probability density assigned to an outcome by a probability model.

How it is used
Supports policy-gradient learning by measuring how likely sampled actions or generated tokens are under a policy.
How deep it goes
Defines log probability and its score-function role, explains numerical benefits, and works through a Gaussian example.
Examples
Uses a 30% action probability and Gaussian actions sampled at 0.82 and 0.3 to illustrate log-probability values.
What you can do after
Evaluate how likely sampled actions or generated tokens are under a policy, and compute token log probabilities from model logits.

Read how Applied Reinforcement Learning explains log probability →

Statistics Every Programmer Needs

Log-likelihood is the logarithm of the likelihood of observed data under a statistical model and its parameter values.

How it is used
Evaluates and compares how well logistic-regression or time-series parameters explain observed data.
How deep it goes
Defines the likelihood transformation and applies it to null-model comparisons, pseudo R², and information criteria without deriving estimation algorithms.
Examples
Uses binary-outcome logistic regression and fitted time-series models as concrete settings for interpreting model fit.
What you can do after
Evaluate model fit and compare fitted models using stored log-likelihoods, pseudo R², AIC, BIC, and HQIC.

Read how Statistics Every Programmer Needs explains log probability →

Machine Learning Algorithms in Depth

A log likelihood is the logarithm of a model likelihood for observed data, commonly optimized instead of the likelihood itself.

How it is used
Uses log likelihood as a classification training objective and as a score for detecting anomalous sensor inputs.
How deep it goes
Derives optimization objectives and gradients for classifiers, then applies Gaussian log likelihood in an implemented anomaly detector.
Examples
Works through logistic regression, Bernoulli naive Bayes document classification, and an LSTM-VAE detecting anomalies in sensor time series.
What you can do after
Build it from scratch as a logistic-regression loss, or apply it via Keras/TensorFlow to threshold anomalous sensor readings.

Read how Machine Learning Algorithms in Depth explains log probability →

Build a DeepSeek Model (From Scratch)

A log probability is the logarithm of the probability assigned to an event, such as a generated token under a language-model policy.

How it is used
Connects sampled language-model tokens to reward-weighted policy-gradient and GRPO updates.
How deep it goes
Progresses from sampled-token intuition to masked tensor scoring and clipped objectives, while deferring production estimators and distributed infrastructure.
Examples
Uses sampled next-token probabilities, completion-level advantages, masked response sequences, and a PyTorch clipped-GRPO loss.
What you can do after
Apply it via a GRPO implementation that scores generated tokens, stores rollout values, masks inputs, and computes clipped policy updates.

Read how Build a DeepSeek Model (From Scratch) explains log probability →

Build AI-Enhanced Web Apps

A log probability is the logarithm of the probability assigned to an event, such as a language-model token.

How it is used
Uses token likelihoods to estimate response confidence and support grounding decisions.
How deep it goes
Explains per-token scores and integration patterns, but stops short of formal probability mathematics and standardized framework support.
Examples
Uses OpenAI configuration, token-score response data, and a confidence-threshold RAG conversation handler.
What you can do after
Apply it via an API configuration to inspect token confidence and trigger grounded-response fallbacks.

Read how Build AI-Enhanced Web Apps explains log probability →

Generative AI in Action

Logprobs are completion metadata containing the natural logarithms of the model probabilities assigned to candidate tokens.

How it is used
Exposes token-level alternatives and confidence signals for debugging prompts, tuning generation, and selecting among completions.
How deep it goes
Defines natural-log token scores and demonstrates response arrays, but does not derive the underlying mathematics.
Examples
Uses a white miniature poodle naming request, Cotton’s token scores, alternatives, and best_of-generated pet-service names.
What you can do after
Diagnose/debug with it by inspecting alternatives, tune randomness, assess uncertainty, and select higher-scoring completions.

Read how Generative AI in Action explains log probability →

Causal AI

A log probability is the natural logarithm of a probability value.

How it is used
Uses logarithmic probabilities when working computationally with canonical probability distributions.
How deep it goes
Defines the transformation, explains its computational motivation, and demonstrates conversion back, but does not derive numerical-stability details.
Examples
Demonstrates Pyro's Bernoulli(0.4), log_prob for outcome 1.0, and math.exp reconstruction.
What you can do after
Apply it via Pyro and Python by evaluating log_prob and exponentiating the result.

Read how Causal AI explains log probability →

Math and Architectures of Deep Learning

The natural logarithm of a probability density or mass value, used to make products and tiny probabilities easier to compute with.

How it is used
Supports probability computation and parameter estimation by replacing difficult likelihood products with simpler additive log-likelihoods.
How deep it goes
Explains the product-to-sum identity and optimization rationale, then applies them to Gaussian estimation without deriving every distribution internally.
Examples
Uses exponential identities, Gaussian likelihood optimization, adult height samples, and PyTorch distribution calculations.
What you can do after
Apply it via PyTorch distributions and optimize Gaussian or other exponential-family likelihoods in logarithmic form.

Read how Math and Architectures of Deep Learning explains log probability →

Reinforcement Learning from Human Feedback

A log probability is the logarithm of the probability assigned to an event or token by a probabilistic model.

How it is used
Computes policy-update signals from token probabilities in reinforcement-learning language-model training.
How deep it goes
Explains token-level log-probability differences, summed completion likelihoods, and their effects under three advantage cases.
Examples
Uses positive, negative, and zero advantages alongside the pg_loss = -advantages * ratio implementation.
What you can do after
Apply it via a policy-gradient loss to reinforce, suppress, or preserve sampled completions.

Read how Reinforcement Learning from Human Feedback explains log probability →

What to know first

Related concepts

Questions about log probability

What is log probability?

A log probability is the logarithm, usually the natural logarithm, of a probability assigned to an event or outcome.

Which Manning books explain log probability?

Build a Reasoning Model (From Scratch), Applied Reinforcement Learning, Statistics Every Programmer Needs, Machine Learning Algorithms in Depth, Build a DeepSeek Model (From Scratch), Build AI-Enhanced Web Apps, Generative AI in Action, Causal AI and 2 more.

What should I know before learning log probability?

Manning books that teach log probability usually introduce mean, parameter, accuracy and gradient first.

What is log probability part of?

Log probability is part of completions api. A completions API is an interface for requesting language-model-generated text from supplied prompts or inputs and configuring generation behavior.

What is log probability also called?

Log probability is also called logprobs.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.