Sequential decision-making
Sequential decision-making is a process of choosing actions across multiple stages in which each action can affect later states, decisions, or outcomes.
Applied Reinforcement Learning
A decision process in which actions are chosen over multiple steps while relevant states, outcomes, or future conditions are not fully known.
- How it is used
- Frames recurring business choices under uncertainty as decisions whose effects shape future conditions.
- How deep it goes
- Connects sequential decision-making to Markov decision processes and solution methods, without detailing their full algorithms here.
- Examples
- Uses business decisions with limited resources and a resource-allocation problem to ground the concept.
- What you can do after
- Evaluate whether reinforcement learning suits a business decision problem and distinguish model-based from model-free approaches.
Read how Applied Reinforcement Learning explains sequential decision-making →
Grokking Bayes
Sequential decision-making is the process of choosing actions over multiple stages while incorporating information and outcomes from earlier stages.
- How it is used
- Extends one-off choices to settings where decisions can respond to observed outcomes.
- How deep it goes
- Works through two-stage policy trees, accumulated costs, terminal utilities, and backward expected-utility propagation.
- Examples
- Uses a government choosing between two costly policies, with the option to stop or implement the other later.
- What you can do after
- Evaluate action sequences by propagating terminal utilities backward and selecting the best follow-up at each stage.
Read how Grokking Bayes explains sequential decision-making →
Timeless Algorithms, The Foundational Ideas
Sequential decision-making is the selection of actions over multiple stages where each action affects later states, options, or outcomes.
- How it is used
- Frames interconnected choices over multiple stages as a sequence to optimize rather than as isolated decisions.
- How deep it goes
- Explains recursive decomposition and solution reuse, while requiring suitable problem structure for smaller solutions to build larger ones.
- What you can do after
- Understand when recursive optimization can produce a globally optimal sequence by reusing subproblem solutions that combine consistently.
Read how Timeless Algorithms, The Foundational Ideas explains sequential decision-making →
Also covered in
- Causal AI
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- Grokking AI Algorithms, Second Edition
What to know first
- reward signalA reward signal is a value or feedback indicating the desirability, benefit, or cost associated with an action, outcome, or behavior.
- actionAn action is a choice or operation selected or performed by an actor that can affect a system, environment, or process.
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- Markov decision processA Markov decision process is a mathematical model of sequential decision-making defined by states, actions, probabilistic transitions, and rewards, with future dynamics depending on the current state and action.
Related concepts
- DQNaddressed by · A deep Q-network (DQN) is a reinforcement-learning method that uses a neural network to approximate the state-action value function in Q-learning.
- dynamic programmingaddressed by · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- reinforcement learningimplemented by · Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
- token generationincludes · Token generation is the iterative process by which a language model produces a sequence of output tokens, typically one token at a time.
- statehas part · State is information describing the condition or configuration of a system or computation at a particular time that may influence its behavior or subsequent transitions.
Questions about sequential decision-making
What is sequential decision-making?
Sequential decision-making is a process of choosing actions across multiple stages in which each action can affect later states, decisions, or outcomes.
Which Manning books explain sequential decision-making?
Applied Reinforcement Learning, Grokking Bayes and Timeless Algorithms, The Foundational Ideas.
What should I know before learning sequential decision-making?
Manning books that teach sequential decision-making usually introduce reward signal, action, AI agent and Markov decision process first.
What kinds of sequential decision-making are there?
Kinds of sequential decision-making include financial trading and token generation.
What are the parts of sequential decision-making?
Sequential decision-making includes state.
What is sequential decision-making also called?
Sequential decision-making is also called sequential decision process.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


