liveBook · Manning

Markov property

As explained in Causal AI

A Markov assumption is the assumption that each variable in an ordered chain depends only on the variable immediately preceding it, conditional on that predecessor.

The Markov property simplifies an ordered probabilistic model by assuming that each variable depends only on its immediate predecessor, given that predecessor. For example, it replaces P(z | x, y) with P(z | y), producing an easier-to-model Markov kernel. section 2.1.5

This is a modeling assumption, not a guaranteed fact about the data: z may still depend on x after conditioning on y, but the remaining dependence is treated as weak enough to ignore. Conditional independence makes the resulting model simpler and more tractable. section 2.1.5 section 2.5

In a Markov decision process, the current state is independent of the full history given the previous state. With actions included, the transition from state s to s′ is represented by P(Sₜ₊₁ = s′ | Sₜ = s, Aₜ = a), the causal Markov kernel. section 12.5.2

Figure 12.28 The Markov decision process represented as a DAG
figure

Use the DAG to trace how the previous state and action determine the next state, and to connect the Markov assumption with causal reasoning. section 12.5.2

How Causal AI covers it

How deep it goes
Explains the factorization simplification, its approximation risk, and its transition-kernel form, without deriving estimation procedures.
Examples
Uses the replacement P(z | x, y) with P(z | y) and an MDP transition P(Sₜ₊₁ = s′ | Sₜ = s, Aₜ = a).
What it assumes
Assumes ordered variables, conditional probability, factorization, and basic causal-graph concepts.
What you can do after
Apply it via simplified conditional distributions and MDP transition functions while recognizing the assumption’s possible approximation error.
How it connects
Connects conditional independence, Markov kernels, causal DAGs, d-separation, do-calculus, and Markov decision processes.

Also explained in

Compare how 4 books explain Markov property →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Causal AI. The section links point to the parts of the book it draws on.