liveBook · Manning

Counterfactual reasoning

Counterfactual reasoning examines hypothetical outcomes under conditions, actions, or policies different from those that actually occurred.

How 2 Manning books explain it

AI Model Evaluation

Counterfactual evaluation estimates outcomes under decisions or policies other than those that generated the observed data.

How it is used
Estimates alternative policy outcomes offline when live experiments are costly, risky, or impractical.
How deep it goes
Explains decision logging, policy-value estimation, assumptions, and stability checks without treating estimates as exact alternate histories.
Examples
Uses movie recommendations and an agent routing requests among four tools to illustrate alternative decisions and outcomes.
What you can do after
Evaluate alternative policies using logged decisions, propensity weighting, baseline comparisons, and reliability checks.

Read how AI Model Evaluation explains counterfactual reasoning →

Causal AI

Counterfactual reasoning is reasoning about hypothetical outcomes under conditions different from those that actually occurred.

How it is used
Uses hypothetical alternatives to attribute outcomes and inform decisions after an action or event has occurred.
How deep it goes
Develops parallel-world representations and a discrete-model inference workflow, then examines why observational and interventional fit cannot guarantee counterfactual accuracy.
Examples
Works through subscriber churn, loan approval changes, and a Monty Hall strategy switch.
What you can do after
Apply it via pgmpy to query discrete hypothetical outcomes conditional on observed events, while checking the assumptions behind the result.

Read how Causal AI explains counterfactual reasoning →

What to know first

Related concepts

Questions about counterfactual reasoning

What is counterfactual reasoning?

Counterfactual reasoning examines hypothetical outcomes under conditions, actions, or policies different from those that actually occurred.

Which Manning books explain counterfactual reasoning?

AI Model Evaluation and Causal AI.

What should I know before learning counterfactual reasoning?

Manning books that teach counterfactual reasoning usually introduce language model, bias, causal inference and policy first.

What is the difference between counterfactual reasoning and A/B test?

Counterfactual reasoning examines hypothetical outcomes under conditions, actions, or policies different from those that actually occurred. An A/B test is a controlled experiment that compares outcomes for two variants assigned to different users or experimental units.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.