liveBook · Manning

Sequential decision-making

Also called sequential decision process

Sequential decision-making is a process of choosing actions across multiple stages in which each action can affect later states, decisions, or outcomes.

How 3 Manning books explain it

Applied Reinforcement Learning

A decision process in which actions are chosen over multiple steps while relevant states, outcomes, or future conditions are not fully known.

How it is used
Frames recurring business choices under uncertainty as decisions whose effects shape future conditions.
How deep it goes
Connects sequential decision-making to Markov decision processes and solution methods, without detailing their full algorithms here.
Examples
Uses business decisions with limited resources and a resource-allocation problem to ground the concept.
What you can do after
Evaluate whether reinforcement learning suits a business decision problem and distinguish model-based from model-free approaches.

Read how Applied Reinforcement Learning explains sequential decision-making →

Grokking Bayes

Sequential decision-making is the process of choosing actions over multiple stages while incorporating information and outcomes from earlier stages.

How it is used
Extends one-off choices to settings where decisions can respond to observed outcomes.
How deep it goes
Works through two-stage policy trees, accumulated costs, terminal utilities, and backward expected-utility propagation.
Examples
Uses a government choosing between two costly policies, with the option to stop or implement the other later.
What you can do after
Evaluate action sequences by propagating terminal utilities backward and selecting the best follow-up at each stage.

Read how Grokking Bayes explains sequential decision-making →

Timeless Algorithms, The Foundational Ideas

Sequential decision-making is the selection of actions over multiple stages where each action affects later states, options, or outcomes.

How it is used
Frames interconnected choices over multiple stages as a sequence to optimize rather than as isolated decisions.
How deep it goes
Explains recursive decomposition and solution reuse, while requiring suitable problem structure for smaller solutions to build larger ones.
What you can do after
Understand when recursive optimization can produce a globally optimal sequence by reusing subproblem solutions that combine consistently.

Read how Timeless Algorithms, The Foundational Ideas explains sequential decision-making →

Also covered in

What to know first

Related concepts

Questions about sequential decision-making

What is sequential decision-making?

Sequential decision-making is a process of choosing actions across multiple stages in which each action can affect later states, decisions, or outcomes.

Which Manning books explain sequential decision-making?

Applied Reinforcement Learning, Grokking Bayes and Timeless Algorithms, The Foundational Ideas.

What should I know before learning sequential decision-making?

Manning books that teach sequential decision-making usually introduce reward signal, action, AI agent and Markov decision process first.

What kinds of sequential decision-making are there?

Kinds of sequential decision-making include financial trading and token generation.

What are the parts of sequential decision-making?

Sequential decision-making includes state.

What is sequential decision-making also called?

Sequential decision-making is also called sequential decision process.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.