liveBook · Manning

Markov chain

As explained in Timeless Algorithms, The Foundational Ideas

A Markov chain is a stochastic model of transitions among states in which the next state depends only on the current state.

A Markov chain models transitions among states under a specific assumption: the next state depends only on the current state. The probability of a sequence of transitions is found by combining the conditional probabilities at each step. section 2.4.2

This makes Markov chains a simple example of how the multiplication rule handles dependent events unfolding over time. The time interval can be chosen to fit the problem, from days to years. section 2.4.2

Applications include:

  • Customer behavior and movement among competing brands. section 2.4.2
  • Website navigation, weather forecasting, and market share analysis. section 2.4.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines the current-state dependence and explains sequence probabilities through successive conditional probabilities.
Examples
Uses customer behavior, brand migration, website navigation, weather forecasting, and market share analysis as applications.
What you can do after
Understand how conditional transition probabilities combine to describe a system's evolution.
How it connects
Connects Markov chains to conditional probability and the multiplication rule.

Also explained in

Compare how 7 books explain Markov chain →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.