liveBook · Manning

State transition

As explained in Applied Reinforcement Learning

A state transition is a change from one environment state to another resulting from an action and possibly random effects.

A state transition is a system’s move from its current state to another after an action, potentially shaped by random effects. Transitions can be deterministic or stochastic, with probabilities used to describe uncertain outcomes. section 2.1

Two transition patterns illustrate how system evolution can be modeled:

  • Deterministic: the current state determines exactly what happens next. section 2.1
  • Stochastic with known transitions: outcomes vary, but probabilities describe how likely each move between states is; data can help estimate them. section 2.1

Daily pricing

In an online store, a chosen price affects customer purchases; sales change tomorrow’s inventory and may influence demand trends. Those effects connect today’s action to the next state. section 2.4

Figure 2.1 Three different types on how a system can evolve moving from one state to another.

Compare the depicted ways a system can evolve from one state to another. section 2.1

How Applied Reinforcement Learning covers it

How deep it goes
Distinguishes deterministic evolution from stochastic transitions with known probabilities and applies transitions in business examples.
Examples
Uses customer plan changes, online-store pricing, and call-center staffing to illustrate system evolution.
What you can do after
Evaluate how business actions affect next states and identify missing information that weakens prediction.
How it connects
Links transitions to states, actions, rewards, Markov chains, and Markov decision processes.

Also explained in

Compare how 12 books explain state transition →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.