An initial state distribution is the probability distribution sampled to choose the first state of a stochastic process. In a Markov process, that state starts the sequence; later states come from a transition function applied to the current state. section 16.1 section 16.6
Its role becomes clearer when the process is built around distributions:
- Choose the first state by sampling the initial distribution. section 16.6
- Pass each sampled state to a transition function, which supplies the distribution for the next state. section 16.1
- Select an initial distribution whose support can reach the target states; Metropolis sampling can use a prior when sampling a posterior. section 16.5
Gambler’s ruin
A gambler begins with a uniformly selected four, five, six, or seven chips. That distribution supplies the first state; subsequent states add or remove one chip until zero or ten ends the sequence. The same pattern can initialize random text with the distribution of first words in sentences. section 16.1 section 16.2
Listing 16.1 A general-purpose Markov process
sealed class Markov<T> : IDistribution<IEnumerable<T>>
{
private readonly IDistribution<T> initial;
private readonly Func<T, IDistribution<T>> transition;
public static Markov<T> Distribution(
IDistribution<T> initial,
Func<T, IDistribution<T>> transition) =>
new Markov<T>(initial, transition);
private Markov(
IDistribution<T> initial,
Func<T, IDistribution<T>> transition)
{
this.initial = initial;
this.transition = transition;
}
public IEnumerable<T> Sample()
{
var distribution = initial;
while (distribution is not Empty<T>)
{
var state = distribution.Sample();
yield return state;
distribution = transition(state);
}
}
}
The implementation stores the initial distribution separately from the transition function, then samples it before entering the transition loop. section 16.1
