An action is both an event caused by surrounding conditions and an intervention chosen to change them. The correct interpretation depends on the question: observed choices support level-1 reasoning, while asking what happens after choosing an option requires a level-2 intervention. section 12.2.1
In a Markov decision process, an action is something an agent can do in a state, producing a change of state. Available actions may depend on the current state, and their consequences are represented through transitions and rewards. section 12.5.1
Sequential decisions add temporal structure:
- A current context can drive the action taken. section 12.3.3
- The action can change the next context, which influences the next action. section 12.3.3
- The ultimate outcome and utility may depend on the entire sequence rather than immediate feedback. section 12.3.3
Grid-world navigation
An agent chooses up, down, left, or right from its current cell. It seeks the goal at {0, 3}, avoids the loss at {1, 3}, and routes around the obstacle at {1, 1}. section 12.5.1
Figure 12.27 A simple grid world
Use the grid-world diagram to inspect how available movements, obstacles, terminal rewards, and action sequences form a decision problem. section 12.5.1
