A state is a data structure describing a possible solution or the environment’s configuration at a particular point in an agent’s interaction. Its meaning depends on the problem: it can encode selected items in a knapsack or record an agent’s current location in a simulator. section 4.4 section 10.5
States support two practical workflows:
- Encode a candidate solution with a fixed representation. In the Knapsack Problem, each binary position marks an item as excluded (0) or included (1); a collection of states forms a population. section 4.4
- Track interaction with an environment. A simulator resets the starting state, returns the current state, applies actions, calculates rewards, and checks whether the goal is achieved. section 10.5
A parking-lot state
A simple simulator stores the agent’s position as coordinates. Applying an action proposes a new position, checks whether it is within bounds, updates the coordinates when valid, and assigns a reward based on the destination cell. The state can then be encoded as a single value derived from the grid dimensions and the agent’s coordinates. section 10.5
Figure 4.11 Binary-encoding the Knapsack Problem
Visualizes how fixed binary positions represent included and excluded items in a Knapsack state. section 4.4
class Simulator:
def __init__(self, road, road_size_x, road_size_y,
➥agent_start_x, agent_start_y):
self.road = road
self.road_size_x = road_size_x
self.road_size_y = road_size_y
self.agent_x = agent_start_x
self.agent_y = agent_start_y #A
def move_agent(self, action):
next_x, next_y = self.agent_x, self.agent_y
if action == COMMAND_NORTH: #B
next_x -= 1
elif action == COMMAND_SOUTH:
next_x += 1
elif action == COMMAND_EAST:
next_y += 1
elif action == COMMAND_WEST:
next_y -= 1
reward_update = 0
if self.is_within_bounds(next_x, next_y): #C
reward_update = self.cost_movement(next_x, next_y) #D
self.agent_x, self.agent_y = next_x, next_y
else:
reward_update = ROAD_OUT_OF_BOUNDS_REWARD #E
return reward_update
Shows state updates when a simulator applies movement actions and handles bounds and rewards. section 10.5
