Markov property
The Markov property is the condition that, given the current state, the distribution of future states is independent of the process’s earlier history.
Causal AI
A Markov assumption is the assumption that each variable in an ordered chain depends only on the variable immediately preceding it, conditional on that predecessor.
- How it is used
- Simplifies sequential probability models and supports causal reasoning in graphical models and Markov decision processes.
- How deep it goes
- Explains the factorization simplification, its approximation risk, and its transition-kernel form, without deriving estimation procedures.
- Examples
- Uses the replacement P(z | x, y) with P(z | y) and an MDP transition P(Sₜ₊₁ = s′ | Sₜ = s, Aₜ = a).
- What you can do after
- Apply it via simplified conditional distributions and MDP transition functions while recognizing the assumption’s possible approximation error.
Build a Robo-Advisor with Python (From Scratch)
The Markov property is the condition that the probabilities of future states depend on the current state and action but not on earlier history.
- How it is used
- Frames financial planning as a reinforcement-learning problem whose transition probabilities use current wealth and asset allocation.
- How deep it goes
- Defines the history-independence assumption and applies it to wealth transitions, while deferring transition-probability implementation to subsequent code.
- Examples
- Uses chessboard positions and wealth under asset allocation to illustrate history-independent transitions.
- What you can do after
- Builds transition models that calculate next-period wealth probabilities from current wealth and selected asset allocation.
Read how Build a Robo-Advisor with Python (From Scratch) explains Markov property →
Optimization Algorithms: AI techniques for design, planning, and control problems
The property of a stochastic process whereby the distribution of the next state depends only on the current state and not on earlier states.
- How it is used
- Explains state transitions in simulated annealing and other stochastic models where only the current state determines the next move.
- How deep it goes
- Defines memorylessness, illustrates transition probabilities, and connects the idea to implementation without developing a formal proof.
- Examples
- Uses transitions among states A, B, and C plus improving and non-improving candidate solutions in simulated annealing.
- What you can do after
- Evaluate/understand whether a stochastic state transition depends only on the currently observed state.
Related concepts
- causal Markov propertyequivalent to · The causal Markov property states that each variable is conditionally independent of its nondescendants given its causal parents in a directed acyclic graph.
- dynamic programminguses · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- Markov chainconfigures · A Markov chain is a stochastic process of states whose transition probabilities depend on the current state rather than the full preceding history.
- transition probabilityconfigures · A transition probability is the conditional probability that a stochastic process moves to a specified next state, possibly given its current state and an action.
- statistical modelingconfigures · Statistical modeling is the use of mathematical or probabilistic models to represent patterns, relationships, or variability in data for description, inference, or prediction.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.