liveBook · Manning

Markov chain

As explained in Statistics Every Programmer Needs

Markov analysis is a technique for modeling transitions among a finite set of states using transition probabilities to predict future states.

A Markov chain models movement among a finite set of states. Its defining assumption is the Markov property: the next state depends on the present state, not the sequence of earlier states. With transition probabilities, it can forecast outcomes such as customer switching, market share, or movement between loan-delinquency categories. section 11.0 section 11.1

Start by defining states that are mutually exclusive and collectively exhaustive: each observation belongs to one state, and no possible condition is omitted. For a customer-switching model, the states can be the available carriers; a state-probability vector gives each carrier’s current share. section 11.2 section 11.2.2

Place period-to-period probabilities in a square transition matrix P. Entry Pᵢⱼ is the probability of moving from current state i to next state j; each row must account for all possible next states. Using a row vector πₜ for the current distribution, calculate πₜ₊₁ = πₜP. Repeat the multiplication for later periods, or use πₜ₊ₙ = πₜPⁿ. section 11.2.2 section 11.5

Forecasting carrier shares

Suppose four carriers have current shares [0.40, 0.35, 0.20, 0.05]. With the example transition matrix, their next-period shares are [0.394, 0.329, 0.217, 0.060]. The first share, 0.394, combines customers retained by the first carrier with customers switching to it: 0.40×0.85 + 0.35×0.08 + 0.20×0.10 + 0.05×0.12. One more multiplication gives [0.39012, 0.31454, 0.22803, 0.06731]. section 11.2.1 section 11.2.2

>>> import numpy as np

>>> pi_0 = np.array([0.40, 0.35, 0.20, 0.05])

>>> P = np.array([
>>>     [0.85, 0.06, 0.07, 0.02],
>>>     [0.08, 0.80, 0.10, 0.02],
>>>     [0.10, 0.10, 0.75, 0.05],
>>>     [0.12, 0.10, 0.08, 0.70]
>>> ])

The NumPy listing defines the initial four-carrier state vector and its transition matrix; use them to reproduce the forecast. section 11.2.2

>>> pi_1 = np.dot(pi_0, P)
>>> print(‘Market shares for period 1:’)
>>> print(pi_1)
Market shares for period 1:
[0.394 0.329 0.217 0.06 ]

This listing multiplies the vector by the matrix and displays the one-period market-share forecast. section 11.2.2

For long-run interpretation, an equilibrium distribution stays unchanged after another transition: π* = π*P. An absorbing state is different: once entered, it cannot be left, making it useful for terminal outcomes such as default or churn. Before extending a forecast, check that its states still cover the system and that transition probabilities and the system’s makeup remain stable. A merger, new entrant, or changing switching pattern undermines the original projection. section 11.1 section 11.2.1 section 11.5

How Statistics Every Programmer Needs covers it

How deep it goes
Works through state definitions, transition matrices, NumPy forecasts, equilibrium, and absorbing states without pursuing abstract proofs.
Examples
Uses four-carrier market shares, a running-or-not-running engine, and loan-delinquency categories.
What it assumes
Builds on introductory probability while introducing Python progressively rather than assuming advanced Python skills.
What you can do after
Apply it via NumPy to calculate one- and two-period state distributions and interpret longer-run behavior.
How it connects
Links the Markov property to conditional probabilities, matrix multiplication, equilibrium distributions, and absorbing states.

Also explained in

Compare how 4 books explain Markov chain →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Statistics Every Programmer Needs. The section links point to the parts of the book it draws on.