A model-free method learns to choose actions from experience without an explicit model of how the environment changes. Each experience provides a state, action, reward, and next state; the agent uses these sampled transitions rather than known transition probabilities. section 4.0 section 6.7
This approach fits problems where the environment’s rules are hidden or difficult to obtain. It avoids requiring a complete transition model, but relies on more interactions, computation, and time; rewards remain essential. section 4.0 section 4.5
Without transition dynamics, state values alone do not identify which action to take. Model-free agents can learn action values, Q(s,a), directly from sampled experience. Each sample is noisy evidence, and tracking values for state–action pairs makes learning more complex. section 6.3 section 6.3
Temporal-difference learning, Q-learning, and SARSA are model-free methods. SARSA is on-policy, while Q-learning is off-policy. section 6.7


