liveBook · Manning

Sequential decision-making

As explained in Applied Reinforcement Learning

A decision process in which actions are chosen over multiple steps while relevant states, outcomes, or future conditions are not fully known.

Sequential decision-making means choosing repeatedly as conditions change and outcomes remain uncertain. Each choice can shape the situation faced by later decisions, so good decisions account for both current information and available resources. section 1.0

Reinforcement learning is one approach: an agent acts, receives feedback, and adapts its strategy through experience. It is not a fit for every problem; limited data or feedback, or a need for simpler and more interpretable methods, may favor other tools. section 1.7

Figure 1.7 Overview of reinforcement learning framework.

Use the framework overview to see how reinforcement learning supports sequential decisions. section 1.7

A Markov decision process provides a framework for formulating these problems. Solution methods differ in whether they use a model of transitions and rewards or learn from experience; dynamic programming applies when the transition model is known. section 4.0

Reinforcement learning can extend classical optimization with adaptive decisions in dynamic settings, but it brings training costs, data needs, and explainability challenges. section 1.9

How Applied Reinforcement Learning covers it

How deep it goes
Connects sequential decision-making to Markov decision processes and solution methods, without detailing their full algorithms here.
Examples
Uses business decisions with limited resources and a resource-allocation problem to ground the concept.
What you can do after
Evaluate whether reinforcement learning suits a business decision problem and distinguish model-based from model-free approaches.
How it connects
Connects sequential decision-making to reinforcement learning, Markov decision processes, dynamic programming, and optimization.

Also explained in

Compare how 3 books explain sequential decision-making →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.