A transition probability measures the conditional likelihood of moving from one state to another during a specified period. In a Markov model, the future state depends only on the present state, so these probabilities form the transition matrix used to forecast changing systems such as markets, machines, and delinquency statuses. section 11.0 section 11.3
Use transition probabilities to:
- Represent the probability of each destination state given a current state; mutually exclusive and exhaustive destinations have probabilities summing to one. section 11.3
- Forecast future state probabilities by multiplying the current probability vector by a constant transition matrix for each period. section 11.2.1 section 11.3
- Model end states by assigning an absorbing state a probability of one for remaining in place and zero for every departure. section 11.4
Engine operating states
For an engine that is running, the probability of remaining operational next week is 0.90 and the probability of failing is 0.10. From the failed state, the probability of self-correction is 0.20 and the probability of remaining failed is 0.80. Repeated matrix multiplication produces an 83% probability of running and a 17% probability of not running after two periods, while longer-run behavior approaches an equilibrium distribution. section 11.3
>>> P = np.array([
>>> [1, 0, 0, 0],
>>> [0, 1, 0, 0],
>>> [0.80, 0, 0.10, 0.10],
>>> [0.30, 0.10, 0.30, 0.30]
>>> ]) #1
>>> I = np.array([
>>> [1, 0],
>>> [0, 1]
>>> ]) #2
>>> A = np.array([
>>> [0.80, 0],
>>> [0.30, 0.10]
>>> ]) #3
>>> B = np.array([
>>> [0.10, 0.10],
>>> [0.30, 0.30]
>>> ]) #4
>>> I_minus_B = I – B #5
>>> F = np.linalg.inv(I_minus_B) #6
>>> FA = np.dot(F, A) #7
>>> print('The final matrix FA is:')
>>> print(FA) #8
The final matrix FA is:
[[0.98333333 0.01666667]
[0.85 0.15 ]]
Inspect the NumPy implementation that partitions a transition matrix, computes the fundamental matrix, and derives probabilities of reaching absorbing states. section 11.4.3
