Uncertainty
Uncertainty is incomplete knowledge or limited confidence about a value, state, interpretation, behavior, or outcome.
Timeless Algorithms, The Foundational Ideas
Uncertainty is the lack of complete information about the state of a system, a quantity, or the outcome of an event.
- How it is used
- Frames learning and decision-making as using incomplete evidence to narrow unresolved possibilities under constraints.
- How deep it goes
- Distinguishes average entropy from information in an outcome and known-model uncertainty from uncertainty about the model.
- Examples
- Uses fair and biased coin flips, a positive diagnostic test, and a robot navigating an unfamiliar environment.
- What you can do after
- Evaluate and understand how evidence reduces uncertainty without necessarily justifying certainty or eliminating decision errors.
Read how Timeless Algorithms, The Foundational Ideas explains uncertainty →
Collaborative Software Design: How to facilitate domain modeling decisions
Uncertainty is the lack of knowledge or trust in the information available when making a decision.
- How it is used
- Guides collaborative design decisions by exposing information gaps and directing investigation toward useful hotspots.
- How deep it goes
- Defines uncertainty, distinguishes it from risk, and gives confidence tracking and feedback-loop practices without formal probability analysis.
- Examples
- Uses coin flipping to contrast risk and tracks a participant’s confidence while exploring EventStorm hotspots.
- What you can do after
- Apply it via collaborative modeling by marking hotspots, monitoring confidence, and revising models through feedback loops.
Grokking Bayes
Uncertainty is lack of complete knowledge about the state of the world, outcomes, or relevant quantities.
- How it is used
- Supports predictions and decisions by representing what is unknown rather than reporting only a single answer.
- How deep it goes
- Distinguishes sources of predictive uncertainty and compares standard deviation with entropy, without developing an entropy calculation here.
- Examples
- Uses weather predictions, a tea-preference posterior, and neural-network predictions beyond observed data.
- What you can do after
- Evaluate/understand when a point prediction hides uncertainty and when standard deviation is an unsuitable measure.
Introduction to Generative AI, Second Edition
Uncertainty is the lack of confidence or knowledge about whether a model output or conclusion is correct.
- How it is used
- Highlights why recognizing uncertainty matters when language models answer high-risk medical questions.
- How deep it goes
- Contrasts accuracy with recognition of knowledge limits, without explaining techniques for modeling uncertainty.
- Examples
- Uses GPT-4’s radiology exam responses and a comparison between model accuracy and a clinician’s recognition of knowledge limits.
- What you can do after
- Evaluate whether confident language and accuracy figures provide enough assurance for high-risk medical use.
Read how Introduction to Generative AI, Second Edition explains uncertainty →
Software Architecture Decision-Making
Uncertainty is incomplete or changing knowledge about requirements, constraints, stakeholders, priorities, or future conditions that affect architectural choices.
- How it is used
- Frames architecture as judgment under changing requirements, constraints, team capability, and delivery pressure.
- How deep it goes
- Explains uncertainty through trade-offs, reversible commitments, feedback, and an uncertain greenfield scenario rather than implementation internals.
- Examples
- Uses a greenfield system with unknown demand and integrations, plus a conservative alternative to operationally uncertain modernization.
- What you can do after
- Evaluate/understand when to commit, defer, or revisit architectural choices as evidence and organizational conditions change.
Read how Software Architecture Decision-Making explains uncertainty →
Architecting Autonomous Agentic Systems
A condition in which the outcomes, costs, benefits, or behavior of a system cannot be known with complete confidence in advance.
- How it is used
- Frames delegated authority as an operational and economic decision under uncertain system behavior and outcomes.
- How deep it goes
- Applies thresholds and three risk mechanisms, including expected failure cost and timing sensitivity, to oversight and business-case decisions.
- Examples
- Uses a hospital agent’s rising uncertainty score and a financial-services data-handling error to illustrate operational and autonomy-scope risks.
- What you can do after
- Evaluate uncertain autonomous initiatives using failure cost, benefit timing, realization curves, and explicit oversight boundaries.
Read how Architecting Autonomous Agentic Systems explains uncertainty →
Statistics Every Programmer Needs
Uncertainty is the condition in which information, outcomes, or underlying quantities are not completely known.
- How it is used
- Frames uncertainty as the reason to use statistics and quantitative methods for forecasting, risk assessment, and structured decisions.
- How deep it goes
- Defines uncertainty and demonstrates Monte Carlo interpretation, but stops short of formal probability theory or mathematical proofs.
- Examples
- Uses call-center staffing, Poisson outcomes, and simulated January 2024 GM stock-price trajectories and summary statistics.
- What you can do after
- Apply it via Monte Carlo simulations to compare outcomes, quantify variability, assess risk, and plan under uncertain conditions.
Read how Statistics Every Programmer Needs explains uncertainty →
Grokking AI Algorithms, Second Edition
Uncertainty is the degree of unpredictability or class mixture in a dataset.
- How it is used
- Guides decision trees toward questions that separate labeled examples into clearer classes.
- How deep it goes
- Calculates uncertainty from label counts and probabilities, then uses weighted reductions to compare candidate splits.
- Examples
- Uses Okay and Perfect diamond labels, Carat and Price thresholds, and Python functions for counting, Gini, and information gain.
- What you can do after
- Evaluate candidate decision-tree questions by calculating class impurity and information gain.
Read how Grokking AI Algorithms, Second Edition explains uncertainty →
Bayesian Optimization in Action
Uncertainty is the lack of certainty about the value or behavior of an unknown quantity or function.
- How it is used
- Guides hyperparameter experiments and BayesOpt decisions when evaluations are expensive and untested outcomes remain uncertain.
- How deep it goes
- Develops intuition through hyperparameter and search examples, then connects uncertainty to Gaussian-process predictions and entropy-based information gain.
- Examples
- Uses neural-network depth trials, a one-dimensional Gaussian-process search, and a lost-remote search to illustrate uncertainty and its reduction.
- What you can do after
- Evaluate where to sample next by weighing uncertain regions against locations with stronger predicted performance.
Read how Bayesian Optimization in Action explains uncertainty →
The Art of AI Product Development
Uncertainty is the lack of complete confidence about the value, state, or outcome represented by an AI system.
- How it is used
- Guides AI product discovery, UX design, experimentation, communication, and cross-functional decision-making.
- How deep it goes
- Explains uncertainty through probabilistic behavior, interface safeguards, experimentation, feedback, and collaboration rather than mathematical internals.
- Examples
- Uses ready-aim-fire choices, simulated AI failures, confidence scores, editable outputs, and post-launch user feedback.
- What you can do after
- Apply uncertainty-aware practices to prototype, evaluate, communicate, and improve AI features while supporting informed user control.
Read how The Art of AI Product Development explains uncertainty →
BDD in Action, Second Edition
Uncertainty is incomplete or evolving knowledge about what a software system should accomplish and how it should be built.
- How it is used
- Manages evolving knowledge so teams progressively discover and deliver solutions aligned with underlying business goals.
- How deep it goes
- Explains nonlinear learning and gives BDD practices for feedback, prioritization, decomposition, and delayed commitment.
- Examples
- Uses early feature demonstrations, user and stakeholder feedback, feature decomposition, and a figure depicting nonlinear reduction of unknowns.
- What you can do after
- Apply it via BDD by seeking early feedback, prioritizing learning-rich features, and preserving options until evidence supports commitment.
Read how BDD in Action, Second Edition explains uncertainty →
Machines that Think
Uncertainty is a condition in which the available information does not determine a single certain interpretation or outcome.
- How it is used
- Frames uncertainty as both a motivation for probabilistic inference and a reliability concern in language generation.
- How deep it goes
- Explains competing token probabilities and uncertainty monitoring, while distinguishing a risk signal from verification of truth.
- Examples
- Illustrates unsupported pattern extension with a dolphin classified as a fish and describes a single-pass hallucination-detection method.
- What you can do after
- Evaluate/understand uncertainty signals as indicators of outputs that may need verification, not as proof that claims are true or false.
Build AI Drug Discovery Pipelines
Uncertainty is a quantitative or qualitative representation of limited confidence in a model prediction.
- How it is used
- Guides active-learning acquisition toward unlabeled molecules about which the current virtual-screening model is least confident.
- How deep it goes
- Defines several estimation strategies and demonstrates stochastic sampling, but defers formal Gaussian-process theory.
- Examples
- Uses predictive variance, ensemble disagreement, dropout sampling, decision boundaries, and repeated neural-network predictions over a molecule pool.
- What you can do after
- Apply it via uncertainty sampling to select molecules for expensive docking or experimental labeling.
Read how Build AI Drug Discovery Pipelines explains uncertainty →
Knowledge Graphs and LLMs in Action
Uncertainty is the lack of complete confidence that information, statements, or conclusions are true or precise.
- How it is used
- Frames uncertainty as a central reliability problem for intelligent-system reasoning and decision support.
- How deep it goes
- Contrasts deductive, inductive, and probabilistic reasoning, but does not formalize uncertainty mathematically.
- Examples
- Uses spam filtering and ambiguous medical diagnosis to illustrate generalization, missing information, and probabilistic recommendations.
- What you can do after
- Evaluate/understand how reasoning strategies handle incomplete, ambiguous, or unreliable information.
Read how Knowledge Graphs and LLMs in Action explains uncertainty →
Grokking Statistics
The lack of certainty about a value, outcome, or conclusion inferred from incomplete or variable data.
- How it is used
- Frames probability, sampling, and statistical inference as tools for reasoning with incomplete or variable data.
- How deep it goes
- Develops probability distributions, sampling, confidence intervals, and the central limit theorem, without formal derivations or advanced computation.
- Examples
- Uses deterministic algorithms, population samples, walking speed, confidence intervals, and predictive-model evaluation.
- What you can do after
- Evaluate estimates and claims using probability distributions, confidence intervals, representative samples, and bias-aware interpretation.
Causal AI
Uncertainty is the state of incomplete or imperfect knowledge about the value, state, or outcome of a variable or process.
- How it is used
- Frames action selection, causal inference, and policy evaluation when outcomes remain uncertain after an intervention.
- How deep it goes
- Connects uncertainty to causal factors, subjective probability, and counterfactuals but stops short of a probability-modeling procedure.
- Examples
- Uses business investment, economic conditions, investor choices, and rejected-option outcomes as concrete cases.
- What you can do after
- Evaluate decisions and policies by modeling uncertain outcomes, causal influences, and counterfactual alternatives with probability.
Designing AI Agents
Uncertainty is the lack of complete predictability about a system's state, behavior, or outcome.
- How it is used
- Frames agent architecture as bounded resource allocation under unpredictable model behavior.
- How deep it goes
- Distinguishes three forms of uncertainty and their architectural implications without specifying a detailed implementation.
- Examples
- Illustrates uncertainty with variable model responses, reasoning-and-action loops, changing web pages, and evolving codebases.
- What you can do after
- Evaluate agent architectures by identifying output, behavioral, and environmental uncertainty.
Also covered in
- Experimentation for Engineers
- Starting Data Analytics with Generative AI and Python
- Deep Learning with R, Third Edition
- Inside AI
- AI GovernanceIn early access (MEAP)
- The Coder CafeIn early access (MEAP)
- Decision EngineeringIn early access (MEAP)
- Architecture Modernization: Socio-technical alignment of software, strategy, and structure
- Machine Learning Algorithms in Depth
- LLM Evaluation and Alignment, The Foundational IdeasIn early access (MEAP)
- The Well-Grounded Data Analyst
- AI Recommender SystemsIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- hallucinationA hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
Related concepts
- entropymeasured by · Entropy is an information-theoretic measure of uncertainty in a probability distribution, defined as the expected value of the negative logarithm of outcome probability.
- cross entropy lossmeasured by · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- informationmeasured by · Information is content that conveys meaningful distinctions or reduces uncertainty about possible states, observations, or outcomes.
- probability theorymeasured by · Probability theory is the mathematical framework for quantifying and reasoning about uncertainty in events, variables, and outcomes.
- epistemic uncertaintyincludes · Epistemic uncertainty is uncertainty in a model’s predictions arising from limited knowledge about the system or data being modeled.
Questions about uncertainty
What is uncertainty?
Uncertainty is incomplete knowledge or limited confidence about a value, state, interpretation, behavior, or outcome.
Which Manning books explain uncertainty?
Timeless Algorithms, The Foundational Ideas, Collaborative Software Design: How to facilitate domain modeling decisions, Grokking Bayes, Introduction to Generative AI, Second Edition, Software Architecture Decision-Making, Architecting Autonomous Agentic Systems, Statistics Every Programmer Needs, Grokking AI Algorithms, Second Edition and 9 more.
What should I know before learning uncertainty?
Manning books that teach uncertainty usually introduce language model, mean, accuracy and hallucination first.
What kinds of uncertainty are there?
Kinds of uncertainty include epistemic uncertainty and retrieval uncertainty.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
















