Decision framework
A decision framework is an organized approach for evaluating alternatives and choosing actions according to goals, criteria, constraints, or evidence.
Timeless Algorithms, The Foundational Ideas
An organized structure of methods and assumptions for evaluating choices and selecting actions.
- How it is used
- Structures choices under uncertainty by linking available acts, unknown states, evidence-gathering experiments, probabilities, and consequences.
- How deep it goes
- Formalizes expected-utility choice, decision-tree rollback, and information value, while recognizing that uncertainty and human judgment remain.
- Examples
- Uses a physician’s decision and lending approval to illustrate how evidence, consequences, and judgment shape choices.
- What you can do after
- Evaluate uncertain choices by defining alternatives and states, weighing consequences by probability, and deciding whether more information is worth acquiring.
Read how Timeless Algorithms, The Foundational Ideas explains decision framework →
Building Reliable AI Systems
A structured approach that guides an AI agent in reasoning, selecting tools and actions, and sequencing tasks.
- How it is used
- Guides agents in selecting tools and ordering actions for multistep requests.
- How deep it goes
- Explains a travel workflow and the ReAct cycle, while distinguishing rule-based from AI-driven approaches.
- Examples
- Uses a flight and hotel booking request to illustrate tool selection, task dependencies, and approval before booking.
- What you can do after
- Evaluate how an agent can select tools and sequence actions for a multistep request.
Read how Building Reliable AI Systems explains decision framework →
AI Model Evaluation
A decision framework is a structured set of criteria and reasoning rules for choosing among actions based on evidence.
- How it is used
- Guides model A/B tests from hypotheses and metrics to rollout, iteration, or discontinuation decisions.
- How deep it goes
- Defines criteria and response thresholds, then adds checks for validity, exposure, segments, guardrails, offline alignment, and practical effect size.
- Examples
- Uses model rollouts, mobile and desktop performance, market differences, and planned versus post-hoc segment findings.
- What you can do after
- Evaluate model experiments and choose whether to launch, iterate and retest, or stop investing in a direction.
Also covered in
- Think Like a CTO
- Building Specialized AI SystemsIn early access (MEAP)
What to know first
- decision makingDecision making is the process of selecting an action, outcome, or judgment from available alternatives based on information, objectives, or constraints.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- experimentationhas part · Experimentation is the planned investigation of a question or hypothesis by conducting trials or interventions and observing their outcomes.
- actionhas part · An action is a choice or operation selected or performed by an actor that can affect a system, environment, or process.
- expected utilityuses · Expected utility is the probability-weighted average of the utility values of possible outcomes associated with an action or decision.
- conceptual stackpart of · A conceptual stack is a layered framework for interpreting the structures and processes operating inside a learning system.
- probabilityuses · Probability is a mathematical measure, from zero to one, of the likelihood that an event or outcome occurs or is true.
- ReActincludes · ReAct is an agent technique that iteratively interleaves reasoning, actions, and observations so results can guide subsequent actions.
Questions about decision framework
What is decision framework?
A decision framework is an organized approach for evaluating alternatives and choosing actions according to goals, criteria, constraints, or evidence.
Which Manning books explain decision framework?
Timeless Algorithms, The Foundational Ideas, Building Reliable AI Systems and AI Model Evaluation.
What should I know before learning decision framework?
Manning books that teach decision framework usually introduce decision making and language model first.
What kinds of decision framework are there?
Kinds of decision framework include ReAct.
What is decision framework part of?
Decision framework is part of conceptual stack. A conceptual stack is a layered framework for interpreting the structures and processes operating inside a learning system.
What are the parts of decision framework?
Decision framework includes experimentation and action.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


