The Markov property simplifies an ordered probabilistic model by assuming that each variable depends only on its immediate predecessor, given that predecessor. For example, it replaces P(z | x, y) with P(z | y), producing an easier-to-model Markov kernel. section 2.1.5
This is a modeling assumption, not a guaranteed fact about the data: z may still depend on x after conditioning on y, but the remaining dependence is treated as weak enough to ignore. Conditional independence makes the resulting model simpler and more tractable. section 2.1.5 section 2.5
In a Markov decision process, the current state is independent of the full history given the previous state. With actions included, the transition from state s to s′ is represented by P(Sₜ₊₁ = s′ | Sₜ = s, Aₜ = a), the causal Markov kernel. section 12.5.2
Figure 12.28 The Markov decision process represented as a DAG
Use the DAG to trace how the previous state and action determine the next state, and to connect the Markov assumption with causal reasoning. section 12.5.2



