liveBook · Manning

Decision framework

A decision framework is an organized approach for evaluating alternatives and choosing actions according to goals, criteria, constraints, or evidence.

How 3 Manning books explain it

Timeless Algorithms, The Foundational Ideas

An organized structure of methods and assumptions for evaluating choices and selecting actions.

How it is used
Structures choices under uncertainty by linking available acts, unknown states, evidence-gathering experiments, probabilities, and consequences.
How deep it goes
Formalizes expected-utility choice, decision-tree rollback, and information value, while recognizing that uncertainty and human judgment remain.
Examples
Uses a physician’s decision and lending approval to illustrate how evidence, consequences, and judgment shape choices.
What you can do after
Evaluate uncertain choices by defining alternatives and states, weighing consequences by probability, and deciding whether more information is worth acquiring.

Read how Timeless Algorithms, The Foundational Ideas explains decision framework →

Building Reliable AI Systems

A structured approach that guides an AI agent in reasoning, selecting tools and actions, and sequencing tasks.

How it is used
Guides agents in selecting tools and ordering actions for multistep requests.
How deep it goes
Explains a travel workflow and the ReAct cycle, while distinguishing rule-based from AI-driven approaches.
Examples
Uses a flight and hotel booking request to illustrate tool selection, task dependencies, and approval before booking.
What you can do after
Evaluate how an agent can select tools and sequence actions for a multistep request.

Read how Building Reliable AI Systems explains decision framework →

AI Model Evaluation

A decision framework is a structured set of criteria and reasoning rules for choosing among actions based on evidence.

How it is used
Guides model A/B tests from hypotheses and metrics to rollout, iteration, or discontinuation decisions.
How deep it goes
Defines criteria and response thresholds, then adds checks for validity, exposure, segments, guardrails, offline alignment, and practical effect size.
Examples
Uses model rollouts, mobile and desktop performance, market differences, and planned versus post-hoc segment findings.
What you can do after
Evaluate model experiments and choose whether to launch, iterate and retest, or stop investing in a direction.

Read how AI Model Evaluation explains decision framework →

Also covered in

What to know first

Related concepts

Questions about decision framework

What is decision framework?

A decision framework is an organized approach for evaluating alternatives and choosing actions according to goals, criteria, constraints, or evidence.

Which Manning books explain decision framework?

Timeless Algorithms, The Foundational Ideas, Building Reliable AI Systems and AI Model Evaluation.

What should I know before learning decision framework?

Manning books that teach decision framework usually introduce decision making and language model first.

What kinds of decision framework are there?

Kinds of decision framework include ReAct.

What is decision framework part of?

Decision framework is part of conceptual stack. A conceptual stack is a layered framework for interpreting the structures and processes operating inside a learning system.

What are the parts of decision framework?

Decision framework includes experimentation and action.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.