Simulation executes a model to reproduce an environment’s interactions or outcomes without acting in the real setting. For reinforcement learning, it provides a controlled sandbox where agents can explore, fail safely, and learn from feedback. section 3.0
Estimating customer transitions
Start a simulated software-service customer at “Buy Pro,” follow transition probabilities, and count the next states. Normalize the counts into probabilities, then compare estimates that include different previous-state histories. section 2.2
Listing 2.1 Simulate Markov Chain
def simulate_markov(P, initial_state, n_steps):
n_states = P.shape[0] #A
chain = np.empty(n_steps, dtype=int) #A
chain[0] = initial_state #B
for t in range(1, n_steps):
chain[t] = np.random.choice(n_states, p=P[chain[t-1]]) #C
return chain
See a Markov-chain simulation that samples each next state from the transition probabilities for the current state. section 2.2










